Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

Struggling to use batch macro to input multiple CSVs with different schema

jt_edin
8 - Asteroid

I gather my problem is very common, and there are solutions here and here that I haven't been able to successfully implement yet.

 

I have dozens of CSV files in a single directory, but they have different schemas. I'd like to open them all using the *.csv wildcard, and then get Alteryx to 'union' the files based on common (or not) field names. I can then reorder the field names with a Select tool.

 

I have downloaded the Batch.yxzp file on this page but I don't know how to get it to work properly. I can read in my directory using the Directory tool, and pick up the FullPath (V_WString) field in the batch macro, but the output returns just one line instead of thousands. Just as the Union tool works, I'd expect all the possible field names along the tops, with <null> values in rows where certain fields are not used.

 

This problem must be routine. Can anyone explain how I need to reconfigure the batch macro (perhaps using the interface designer) to accomplish this task? Thanks

 

 

6 REPLIES 6
pcatterson
11 - Bolide

Here is a macro I made to pull together CSV files.  You can utilize it hopefully and look at the inner workings for a better understanding of how to build your own.

 

The D output is the data, the F output is a list of the files that failed to pull in, the S output is the list of successfully pulled in files.

jt_edin
8 - Asteroid

Thanks very much. What tool do I need to plug into it? Directory doesn't work for me, but I'm probably doing something stupid...

 

dir.PNG

pcatterson
11 - Bolide

You have to connect the Directory to BOTH inputs: the blank one and the upside down question mark one.

 

The macro should look have these selections:

 

A.png

 

B.png

jt_edin
8 - Asteroid

Thanks. Sorry - I'm not able to adapt this to my needs.

 

Say I have three CSV files that have slightly different schemas, eg:

 

File1

IDField1Field2
111

 

File2

IDField1Field3
222

 

 

File3

IDField1Field4
333

 

Here is what I am expecting:

dir1.PNG

 

But here is what your tool is giving me:

dir2.PNG

 

Any ideas to achieve my aim? Thanks again

pcatterson
11 - Bolide

Go inside the macro and change the delimiter character from pipe (|) to comma (,) in the Dynamic Input tool.  That should fix it. 

jt_edin
8 - Asteroid

First class - excellent. I will now mark up this solution. 

Labels