Pulling data using In-database Data stream in without creating temporary table

[rjbrsh]

Hi, 

I am very new to In-db tools. I've used python tool to pull data from the server and been trying to use In-database 'Data Stream In' tool to bring the data in but the problem is I don't have permission to create a temporary table, so it errors out.  Is there any other way the data can be pulled in without having to write in a temp table.

Any help is much appreciated. 

Thank you! 

[gabrielvilella]

Hi @rjbrsh, I believe you are looking for the Dynamic Input In-DB tool, where you can pass the query and connection name.

[apathetichell]

Datastream-In takes data into your In-DB tool system - IE your server. It requires either a permanent table or a temporary table so that you can use it on your server - since this is on server.

If you want to manipulate data in your workflow - you need Datastream Out - which takes data OUT of In-DB and into a standard workflow. Try to filter/summarize/parse this data first. You don't want to be bringing out oodles (10,000,000+) rows depending upon your system and server.

[rjbrsh]

Thank you @gabrielvilella.I thought Dynamic input would be a solution too. My output from the python tool is a table with 4 columns of data, somewhat like in image attached.  If I were to connect the data directly from the server, I would have a connection to db and the sql to get the data but in the case like mine where I have to transform the  data via python prior to pushing it through the input, I lack the knowledge on what would be the best way to go about it. . I looked for some documentation on these but failed to find any. Any insight would be very much appreciated. 

Thank You!

[gabrielvilella]

First you need the Connect In-DB or the Dynamic Input to get the data from the database. Then you use the Data Stream Out tool to pull data from the database. Finally, you can transform this data normally on your workflow.

[rjbrsh]

Hi @gabrielvilella, that makes sense but below is workflow I wanted to create. I was wondering if there was any other way I could pull the data in through in-database tool without having to write on a temporary table. 

[apathetichell]

As I noted in my initial response to your question - you have two options:

1) query the data in a regular workflow - your data on server has to exit via datastream out or be brought in via  standard input data tool.

2) query the data in-db. Your python data must be given an address on your server so your server can use it. This is datastream in. Think about it. In order for your server to be able to use your data it has to be on your server. Temporary/regular table allow your sql platform to find the data.