Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

How to loop through files in batches?

Mond
8 - Asteroid

I have a project where I need to loop through sets of files based on their dates in a directory. Here's how my directory will look like.

 

First File 12-20-2021

Second File 12-19-2021

Third File 12-20-2021

Fourth File 12-20-2021 

First File 12-14-2021

Second File 12-13-2021

Third File 12-14-2021

Fourth File 12-14-2021 

etc...

 

First and Second files all have the same schema, Third and Fourth have different schemas.

 

I am thinking that I can use the directory tool to get all the First Files in the directory and then regex the dates. I would then need to loop through all of the dates and process the files in batches.I am not sure how to do this in the designer though. I would imagine I would have to feed the date to a macro that then goes to the directory and opens the first/second files and combines their data for processing? Then another macro would get the same date and open the third file for separate processing as well as for the Fourth file I guess I would need a different macro?

 

Sorry if this is confusing, I am trying to figure out how to process these four files together and then move on to the next set of files in the most efficient way possible. I'm new to macros 😞

4 REPLIES 4
tsturge
7 - Meteor

2 suggestions

1) download the crew macros, they have some helpful input tools that help get the data in to Alteryx.  Search the community and you should see the download link

 

2) if possible in your business process you might want to organize files by similar header row, it can reduce the complexity

 

This isn't a full solution but can help speed up the data ingestion piece.

 

tsturge_0-1642209520823.png

 

Qiu
20 - Arcturus
20 - Arcturus

@Mond 
I believe you are describing a classic importing Excel sheets/files with different schema.

There is an existing macro here  which can be used for your case.
If you can provide some sample datasets, i can try to modify it to suit your purpose.

 

Mond
8 - Asteroid

thanks for sharing but for reason i'm getting an error to import the workflow.

 

Failure to Import C:\Users\LO\Downloads\Multi Sheet Import Example 2.yxzp.
There was an error opening "C:\Users\LO\Downloads\Multi Sheet Import Example 2\Multi Sheet Import Example 2.yxmd":

Qiu
20 - Arcturus
20 - Arcturus

@Mond 
Sorry I can not tell what is the error exactly. Maybe the version?
Can you upgrade your alteryx to the latest one?

Anyway, I am uploading the files again.

 

Labels