Batch import files with varying schema

I'm trying to build a workflow that will do the following:

- Crawl through a set of directories looking for .csv files that match certain wildcards.

- Return a list of those file paths.

- Import and merge all those files into a single table, adding fields as needed.

I have the first two steps working, currently using the directory tool and a batch macro, but it throws an error whenever it encounters a file that doesn't match the expected schema, saying, "The field schema for the output [Output Name] changed between iterations." Seeing as how that's exactly what I want to work through, I'm thinking I need a different approach.

Any ideas? Thanks in advance.

Input

Preparation

Accepted answers

daniel_mmi

Thanks! I had to dig back through a couple levels of blog posts to find this image, which contained the setting I needed to adjust:

By default the first Output Mode radio button is selected. I picked the second, 'Auto Configure by Name', and the resulting output was exactly what I was looking for. Lots of digging for a simple answer, but thanks for pointing me in the right direction!

All comments

s_pichaipillai

Hi Daniel

Reading diffrent schema in single shot is little interesting

because when you specify a schema for a single file during design, then the workflow look for same structure for all other files

to do this, there is an Macro from for reading all Excel from cameron

have never tested it for CSV file , but take a look at this and modify your Macro for CSV

http://community.alteryx.com/t5/Engine-Works-Blog/The-Ultimate-Alteryx-Holiday-gift-of-2015-Read-ALL-Excel-Macro/ba-p/9988

daniel_mmi

Thanks! I had to dig back through a couple levels of blog posts to find this image, which contained the setting I needed to adjust:

kcoulon

Wanted to do my own and then make sure I understood how to use the tools others were using to get this done, including the macro. Definitely trying to get more comfortable with macros, which are new to me in Alteryx!

Challenge 11 macro.yxmc

challenge_11_solutions.yxmd

challenge_11_solutions_2.yxmd

challenge_11_solutions_3.yxmd

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703