In DataBase Tools

[maxwell123]

Good afternoon everybody,

I am currently work with very large data sets and found out about In-Database tooling as a way to speed up the process.

My questions is, if I am to do a bunch of transformation and cleaning and merges and a whole host of other transformations to the data, does it effect the source of the data. Say for example I was pulling from Snowflake, would all my actions with the data effect Snowflakes tables ie. the source of the data?

Asking because if I am able to transform the data directly with the In-Database tools then just port it back into Alteryx using the "Data Stream In" tool, it would save me almost 14 min of loading time, as opposed to using the "Input Data" tool.

[LindonB]

The in-database tools are very helpful if you'd prefer to visualize or create a workflow rather than read/write all the SQL, but they are essentially just augmenting the SQL. My advice is to aggregate and transform your data as much as possible using either SQL or the In-DB tools before streaming it out.

Generally, those tools are helping write a query and not editing the tables themselves. But as with a SQL query, you can also use it to edit tables. For that, you would use the Write Date In-DB after the in database tools. If you stream data out of the database, you can use a Stream Data In afterwards to overwrite or append data to tables. 

[leefarrell]

When using In-Database tools in Alteryx, your transformations and cleaning operations are done directly within the database (like Snowflake) without affecting the source tables. In other words, the source data remains unchanged.

You can transform and merge data within Snowflake and then bring the results back into Alteryx using the "Data Stream In" tool. This approach should indeed help you save time compared to loading data with the "Input Data" tool.

Let me know if you need any more details!

[apathetichell]

check out dynamic output in-db - and export the query to get a better understanding of this. Alteryx is emplying SQL pushdown where it translates standardize Alteryx tools to SQL langague for your underlying DB. Without a datastream out/write-in-db/browse - this is the same as executing a select ... query on your db - with nothing being pulled into memory on your system.