Output 10M Joined records into multiple excel sheets/files/csv in 1 flow.

[Isha_Gupta]

Hey,

I am Working on a dataset which takes multiple inputs in a workflow, Joins all of them 1 by 1 after certain manipulations.

The final dataset generated is a bit more than 10 million rows and more than 100 columns.

Now I know, it can be output in Alteryx format and many others, But I am looking either Excel or CSV.

I need the workflow to run, keep writing the output to 1 Excel file or worksheet or 1 csv file then as soon as the limit is about to be reached, switch over to the next sheet or file for the remaining records and so on.

Thanks

[apathetichell]

Use RecordId - You would then create a filename/sheet name dynamically  ("myexcelfile_"+tostring(ceil([RecordID]/600000))+".xlsx|Sheet1") or something like that - and then set your output data to change entire filepath based upon whatever filename is assigned to that specific row.

This is not the ideal way to store your data.

[Isha_Gupta]

Can you explain in a more simpler terms. Especially the expression to create dynamic sheets or excel.

[binuacs · Accepted solution]

@Isha_Gupta One way of doing this with the tile tool (equal records property) 

[Isha_Gupta]

Wow, Thanks for the solution. Simpler than I expected it to be.