Struggling to use batch macro to input multiple CSVs with different schema

[jt_edin]

I gather my problem is very common, and there are solutions here and here that I haven't been able to successfully implement yet.

I have dozens of CSV files in a single directory, but they have different schemas. I'd like to open them all using the *.csv wildcard, and then get Alteryx to 'union' the files based on common (or not) field names. I can then reorder the field names with a Select tool.

I have downloaded the Batch.yxzp file on this page but I don't know how to get it to work properly. I can read in my directory using the Directory tool, and pick up the FullPath (V_WString) field in the batch macro, but the output returns just one line instead of thousands. Just as the Union tool works, I'd expect all the possible field names along the tops, with <null> values in rows where certain fields are not used.

This problem must be routine. Can anyone explain how I need to reconfigure the batch macro (perhaps using the interface designer) to accomplish this task? Thanks

[pcatterson]

Here is a macro I made to pull together CSV files.  You can utilize it hopefully and look at the inner workings for a better understanding of how to build your own.

The D output is the data, the F output is a list of the files that failed to pull in, the S output is the list of successfully pulled in files.

[jt_edin]

Thanks very much. What tool do I need to plug into it? Directory doesn't work for me, but I'm probably doing something stupid...

[pcatterson]

You have to connect the Directory to BOTH inputs: the blank one and the upside down question mark one.

The macro should look have these selections:

[jt_edin]

Thanks. Sorry - I'm not able to adapt this to my needs.

Say I have three CSV files that have slightly different schemas, eg:

File1

ID	Field1	Field2
1	1	1

File2

ID	Field1	Field3
2	2	2

File3

ID	Field1	Field4
3	3	3

Here is what I am expecting:

But here is what your tool is giving me:

Any ideas to achieve my aim? Thanks again

[pcatterson · Accepted solution]

Go inside the macro and change the delimiter character from pipe (|) to comma (,) in the Dynamic Input tool.  That should fix it. 

[jt_edin]

First class - excellent. I will now mark up this solution.