Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

Running Batch Macro on Multiple files from a folder

StockMarket
8 - Asteroid

I have gone through many example batch macro threads here, but since I am new to Alteryx, therefor I am finding it a bit complicated to execute correctly in my case. Specially the part where we have to create the "Control Parameter Step" and update it with new file names, so that the process can be repeated on multiple files. I am having trouble trying to find the best way to create a batch macro which will work in my case, so any help or suggestions would be greatly appreciated.


I have to run this workflow on csv data files, which are quite large, having around 4 million rows per csv file and I have to run this flow onto hundreds of such files present within the same folder. The good news is that all these files have the exact same structure and format and a constant schema throughout. So this should reduce the complexity of the batch macro, otherwise some examples of batch macros were looking very complex and hard to understand to me.


I need to make sure that the complete workflow runs on the FIRST CSV FILE and generate the required output files, then the complete workflow runs on the SECOND CSV FILE and generate the required output files and this process continues, till the VERY LAST CSV FILE is processed in the given folder. All csv files are located in this same folder - "C:\Data\Batch Macro Sample Data".

 

I cannot first import all the csv files into a big table and then run the workflow. It has to be done, ONE FILE AT A TIME, separately and keep on repeating, one after the other, till all the files have been processed in that folder.

 

I have gone through a lot of different links, but have failed to create the desired output. I would really appreciate, if someone could take the sample data files that I have attached and create a workflow based on that, instead of just posting here the links to other threads.


To make it easy for you, I have attached three csv files having the sample data, on which this workflow needs to be run. Although the actual data files are quite big, having millions of rows in it, but I have made a very small sample out of it for sharing on the forum. All this data has a constant schema throughout.


I have also attached the Alteryx Worflow Package .yxzp file, so that you can easily open it up on your machine. I have used the latest version of Alteryx for creating this workflow. You may please have a look.


Thanks a lot

 

 

Workflow.png

11 REPLIES 11
AkimasaKajitani
17 - Castor
17 - Castor

Hi @davidnolan 

 

Sorry for the late reply.

 

The setting to be changed is file path.And you need to be careful to the sheets as well.

 

If the Excel files have a specific sheet. The setting is as follows.

 - Action tool should be connected to the formula tool.

 - Take File/Table Name From Field option is "Change Entire File Path" at Output Data tool.

 - Of course, Input Data and Output Data tool should be set using for Excel input/output.

 

 

AkimasaKajitani_3-1674226094025.png

 

 

 

AkimasaKajitani_4-1674226132982.png

 

 

 

AkimasaKajitani_1-1674225402628.png

 

 

davidnolan
7 - Meteor

This has worked perfectly thank you so much @AkimasaKajitani !

Labels