Alteryx Designer Desktop Knowledge Base

Definitive answers from Designer Desktop experts.

Reading multiple CSV files at once

AndrewL
Alteryx
Alteryx
Created

Question

Here is the original question posted in the Forum:

I am working on a project where basically Ineed to use the input data tool to import a csv file for every day of the month. Is there any easier way to do this than to open 30 input data icons then doing a join on all 20 of them? It seems like maybe Icould actually just chose input data one time and highlight all the files Ineed but that didn't work. its just that it takes a long time to drag 30 data inputs in then join them all. They are just CSV files.

Answer

There are several ways to do this - the easiest is to use the wildcard character in an Input Tool (master it here) as suggested one of the responders:

Depending on the file name and location you can even use a standard input tool for this. For example if you have a number of csv's on the the C drive named Sales_20160101.csv, Sales_20160102.csv, ..., Sales_20160309.csv you could enter C:\Sales_*.csv in a standard input tool and all files would be picked up and merged.

In the event the files are named differently, are not all in the same directory, and/or you need to further limit which files are read in, you'll want to use the other method suggested with the Dynamic Input Tool(learn about the tool in detailhere😞

You can use a dynamic input tool to load a set of files.

It takes a list and load them all. It can be configured to include the filename as an entry one very row as well.

There is also a Directory Tool which will search a folder for a list of files.

The combination of the two should allow you to do what you need.

These methods apply to most file types, not just CSV files. The main point in both methods is that in order for either method to work is that each of the files must have the same schema, otherwise you will get read errors.

If your files have different schemas, all is not lost. Take a look at this article for a way to make it work!

Comments
Spatel68
7 - Meteor

Hi,

 

Is there a way to merge multiple .csv files from a folder but with an except clause to exclude a file with certain string or characters?

 

 

DawnDuong
13 - Pulsar
13 - Pulsar

@Spatel68 I would recommend a 3-step approach:

1) Directory tool to get a list of all the file inside a specified directory (incl. sub-directories if you want that)

2) Filter tool + regex to take only the files you want

3) Use batch macro after that

Spatel68
7 - Meteor

Thank you DawnDuong.  I'll try the above mentioned steps.