Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

Batch import files with varying schema

daniel_mmi
9 - Comet

I'm trying to build a workflow that will do the following:

- Crawl through a set of directories looking for .csv files that match certain wildcards.

- Return a list of those file paths.

- Import and merge all those files into a single table, adding fields as needed.

 

I have the first two steps working, currently using the directory tool and a batch macro, but it throws an error whenever it encounters a file that doesn't match the expected schema, saying, "The field schema for the output [Output Name] changed between iterations." Seeing as how that's exactly what I want to work through, I'm thinking I need a different approach.

 

Any ideas? Thanks in advance.

8 REPLIES 8
s_pichaipillai
12 - Quasar

Hi Daniel

Reading diffrent schema in single shot is little interesting 

because when you specify a schema for a single file during design, then the workflow look for same structure for all other files

to do this, there is an Macro from for reading all Excel from cameron

have never tested it for CSV file , but take a look at this and modify your Macro for CSV

 

http://community.alteryx.com/t5/Engine-Works-Blog/The-Ultimate-Alteryx-Holiday-gift-of-2015-Read-ALL...

daniel_mmi
9 - Comet

Thanks! I had to dig back through a couple levels of blog posts to find this image, which contained the setting I needed to adjust:

 

auto_config_by_name.png

 

By default the first Output Mode radio button is selected. I picked the second, 'Auto Configure by Name', and the resulting output was exactly what I was looking for. Lots of digging for a simple answer, but thanks for pointing me in the right direction!

Rene
5 - Atom

Thanks, had the same problem using access files; this post gave the right solution!

dsouthh
7 - Meteor

When I choose the second radio button, my batch macro only runs a single iteration. How did you prevent this from happening? 

dsouthh
7 - Meteor

Nevermind,I connected a union tool to my macro output and it now runs through all iterations. Previously, I had no outgoing connection to the batch macro. 

kcoulon
8 - Asteroid

Wanted to do my own and then make sure I understood how to use the tools others were using to get this done, including the macro.  Definitely trying to get more comfortable with macros, which are new to me in Alteryx!

Naledi
7 - Meteor

@dsouthh wrote:

Nevermind,I connected a union tool to my macro output and it now runs through all iterations. Previously, I had no outgoing connection to the batch macro. 


I just learned that this doesn't have to be a union tool. The union doesn't affect what happens in the macro. It's rather that there has to be some kind of tool following the batch macro in order to show all of the results in the data preview. 

Lilylu11
5 - Atom

How did you do that? Can you share a screenshot? I ran into the same problem

Thanks in advance.

 

Lily

Labels