Happy 8th birthday to the Maveryx Community! Take a walk down memory lane in our birthday blog, and don't miss out on the awesome birthday present that all Maveryx Community members get to take advantage of!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

Merging multiple csv files into a single one

16 - Nebula
16 - Nebula

Hello everyone,


So.... I'm kinda stuck now and don't know how to keep on. I have around 100 csv files in a folder that I need to merge together. The process is as follows:


Read csv files -> Interact with them (formulas etc.) -> Union them all


All the files have the same number of columns, but the columns are named always differently (need also to aggregate the name of the column to the file) and have different number of rows.


I know I should use some macro and maybe a temporary file to keep adding one after another in some iterative way. The thing is that I can't figure it out.


Any ideas?


Thank you in advance!

9 - Comet

I've attached something that might help give you some ideas? Without knowing more about how you want to interact with the individual files, it's hard to tell at what precise point the iteration should happen etc. But maybe some of this will help (or maybe you are already doing this and it's timing the calculations that's the issue?)


The idea is this: process each file through a batch macro. First step, rename the fields with numbers and extract the original field names. Add a "type" field to distinguish between header rows (extracted original field names) and body rows (content). Then union everything together. If you need to do different calculations based on the file, you could add a "file name" field in the batch macro and do the processing after unioning all the files together.


It'll take say two csvs like these



and union them like so


union csv output.PNG

16 - Nebula
16 - Nebula

Yay thats it! That is what I was looking for perfect!


Many thanks :)