Alteryx Designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.

Struggling to use batch macro to input multiple CSVs with different schema

8 - Asteroid

I gather my problem is very common, and there are solutions here and here that I haven't been able to successfully implement yet.


I have dozens of CSV files in a single directory, but they have different schemas. I'd like to open them all using the *.csv wildcard, and then get Alteryx to 'union' the files based on common (or not) field names. I can then reorder the field names with a Select tool.


I have downloaded the Batch.yxzp file on this page but I don't know how to get it to work properly. I can read in my directory using the Directory tool, and pick up the FullPath (V_WString) field in the batch macro, but the output returns just one line instead of thousands. Just as the Union tool works, I'd expect all the possible field names along the tops, with <null> values in rows where certain fields are not used.


This problem must be routine. Can anyone explain how I need to reconfigure the batch macro (perhaps using the interface designer) to accomplish this task? Thanks



11 - Bolide

Here is a macro I made to pull together CSV files.  You can utilize it hopefully and look at the inner workings for a better understanding of how to build your own.


The D output is the data, the F output is a list of the files that failed to pull in, the S output is the list of successfully pulled in files.

8 - Asteroid

Thanks very much. What tool do I need to plug into it? Directory doesn't work for me, but I'm probably doing something stupid...



11 - Bolide

You have to connect the Directory to BOTH inputs: the blank one and the upside down question mark one.


The macro should look have these selections:





8 - Asteroid

Thanks. Sorry - I'm not able to adapt this to my needs.


Say I have three CSV files that have slightly different schemas, eg:












Here is what I am expecting:



But here is what your tool is giving me:



Any ideas to achieve my aim? Thanks again

11 - Bolide

Go inside the macro and change the delimiter character from pipe (|) to comma (,) in the Dynamic Input tool.  That should fix it. 

8 - Asteroid

First class - excellent. I will now mark up this solution.