ETL Process using IN-Database tools in Alteryx

[Bharath]

Hi, 

I am pulling data from Apache Spark since its a big data I am using In-Database tools option in Alteryx. now I need to perform following ETL is it possible to do this in Alteryx ?

[BenMoss · Accepted solution]

Traditionally In Alteryx designer I would achieve this using the generate rows tool; I’m not sure how you could achieve this directly by using the in-dB tools, in fact I’m relatively confident to say you couldn’t .

Just how large is the DB you are working with?

What I would consider doing is using the in-dB summertime tool, grouping by your start date and end date field (thus aggregating the data to the level needed for the generate rows tool to work). Then using the ‘stream out’ tool to bring your aggregated data out of the in-dB tools. The. Use the standard generate rows tool, before using the stream in tool to pull your data back in-dB. You can then use a join tool against your original data stream on the start date and end date fields and you’ll then have the required structure.

I hope this helps.

Ben

[JoshKushner · Accepted solution]

Hi Bharath,

This can be done with the generate rows tool per the below and attached.

Workflow

Input

Results

Generate Rows

[Bharath]

You are really awesome. thank you for your help. It worked really well. I aggregated the data using in-database tools then I used stream out tool to get the stream to use normal ETL tools.