How to open multiple zip files with Input Data Tool?

[SharonNkn]

Does anyone know how to input multiple zip files with Input Data Tool?

I am trying to input 2,300 zip fies(all of them have one csv file in it  and they are in the same folder) with Input Data Tool, but cannot find the way to select multiple zip files in "Open a data file" window.

I know that we can open mulple csv files using " * ", however it did not work for zip files.

Please let me know how to figure it out.

[jdunkerley79 · Accepted solution]

I think you can do this using a directory tool to get a list of all the zip files and then feed it into a dynamic input tool updating the filename as long as the csv files are named the same within the zip.

I have attached a sample showing how it might work. You will need to change the CSV filename in the formula tool if this would work

Otherwise if filenames vary, you could do this by extracting all the files into a temporary folder and importing the results. 

I am happy to put a sample together for the second if that is the case

[shradhaahir]

I have 6 zip files(names differs) within which there are csv file with same name as zip file.

I want to collate all csv file into one file.

Wilcard path is not working. how to use dynamic input here?

[WSData_dup_359]

A little late, but if you are still looking for an answer, check out JD's response above.  As long as your files all of the same field layout, you can use the workflow he provided and simply change the path on the directory and modify the formula to reflect your files.

[JamesChiswell]

Could you please put a sample together for the second case where the filenames vary.

Cheers

James

[TeePee]

Thanks! This was super helpful.  Here's how I tweaked it for reading in multiple .csv.gz files (again, provided they all use the same schema).  It only takes a couple of minutes.  I love standing on the shoulders of giants 🙂

Here's a screenshot of jdunkerley79's flow:

Step One: Directory Tool

• Amend the directory to point to the folder containing the .csv.gz files.  

• Amend the File Specification to say *.csv.gz

 

Step Two: Formula Tool

• Amend the Formula tool text to say this exactly (don't make any other changes): [FullPath] + '|||Query=datafile.csv'

Step Three: Directory Tool

• First, click on the Directory Tool's Edit... button:

• Then point the "Connect to a file..." field to any one of the .csv.gz files in your folder.

• Check the "2 File Format" says Gzip

• Check the "3 File in Archive" says datafile.csv

• If you'd like to check which files have been loaded (eg using a Summarize tool), make sure to select "File Name Only" as the option for "5 Output File Name as Field."  This is useful for troubleshooting; if a file accidentally has a different schema you won't see any other errors or warnings. Optional but highly recommended.
• Check "6 Parse Selected File as" says *.csv
• Finally check over all the other selections to ensure they work with your particular data set (eg does your data have headers? does the data start on the first row? etc)

[Austin004lasrado]

Hello, 

I need a similar feature but my CSV file names are changes (Even ZIP but i can change the ZIP file manually to make it more Static). is there any solution i can use to Import such file via Dynamic tool?

[Austin004lasrado]

Hello @jdunkerley79 ,

Can you please confirm the workflow if the Filenames Vary?

Thanks and Regards,

Austin Lasrado