community
cancel
Showing results for 
Search instead for 
Did you mean: 

Alteryx designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.
SOLVED

Struggling to use batch macro to input multiple CSVs with different schema

Asteroid

I gather my problem is very common, and there are solutions here and here that I haven't been able to successfully implement yet.

 

I have dozens of CSV files in a single directory, but they have different schemas. I'd like to open them all using the *.csv wildcard, and then get Alteryx to 'union' the files based on common (or not) field names. I can then reorder the field names with a Select tool.

 

I have downloaded the Batch.yxzp file on this page but I don't know how to get it to work properly. I can read in my directory using the Directory tool, and pick up the FullPath (V_WString) field in the batch macro, but the output returns just one line instead of thousands. Just as the Union tool works, I'd expect all the possible field names along the tops, with <null> values in rows where certain fields are not used.

 

This problem must be routine. Can anyone explain how I need to reconfigure the batch macro (perhaps using the interface designer) to accomplish this task? Thanks

 

 

ACE Emeritus
ACE Emeritus

Here is a macro I made to pull together CSV files.  You can utilize it hopefully and look at the inner workings for a better understanding of how to build your own.

 

The D output is the data, the F output is a list of the files that failed to pull in, the S output is the list of successfully pulled in files.

Asteroid

Thanks very much. What tool do I need to plug into it? Directory doesn't work for me, but I'm probably doing something stupid...

 

dir.PNG

ACE Emeritus
ACE Emeritus

You have to connect the Directory to BOTH inputs: the blank one and the upside down question mark one.

 

The macro should look have these selections:

 

A.png

 

B.png

Highlighted
Asteroid

Thanks. Sorry - I'm not able to adapt this to my needs.

 

Say I have three CSV files that have slightly different schemas, eg:

 

File1

IDField1Field2
111

 

File2

IDField1Field3
222

 

 

File3

IDField1Field4
333

 

Here is what I am expecting:

dir1.PNG

 

But here is what your tool is giving me:

dir2.PNG

 

Any ideas to achieve my aim? Thanks again

ACE Emeritus
ACE Emeritus

Go inside the macro and change the delimiter character from pipe (|) to comma (,) in the Dynamic Input tool.  That should fix it. 

Asteroid

First class - excellent. I will now mark up this solution. 

Labels