Dynamic reading of different file formats and folders

[verast23]

Hi community,

I have built a workflow to read in different Excel and csv files from different folders (vary each quarter) and then do some processing steps.

Below I have described my workflow and it would be great if you could give me some tips how to improve it. 

So for example:

For Q1 this year, I have three folders A, B and C and in each of these three folders there is one xlsx file and two csv files with different number of fields. 

So I have to read in now the xlsx file and csv files of A, then do some preprocessing steps and output an Excel file which is named e.g. xyz_A.xlsx, then I have to do the same steps with the files in folder B and C. The end user only should copy the file path to the repsective quarter, so where the folders A, B and C are located and then the descirbed steps should be executed.

• So in my workflow I have first a directory tool, to get all subdirectory files in the respective quarter folder.
• Then I filter the "Full Path" name that the Full Path contains "A" to get the files for A. Then I use 3 Dynamic Input tools to read in the Excel file in A as well as the csv files (csv files have different number of fields, therefore I use 2 input tools)
• Then I do the calculation steps, and output the Excel file for A
• I add all the tools to the container and copied the container twice, to do the same for folder B and C, just changed the filter tool that the FullPath contains B and C
• Now when the end user copies the path of folder Q1 to the interface text box and clicks run, the tools in the container are executed one after the other and the respective Excel files are saved.

So the whole workflow does work, but it seems to me that this is not the best or most efficient option.

So do you have an tips how to improve the process of reading in the files and outputting the files?

Thank you in advance!

[Pang_Hee_Choy]

look like the only different is folder.

if that is the case, then you can try to learn about batch macro.

https://community.alteryx.com/t5/Interactive-Lessons/Creating-a-Batch-Macro/ta-p/657923 

[verast23]

So I've tried to create the Batch Macro, but I didn't figred out how to set it up properly..

[apathetichell]

Go back to learning batch macros... post a screen grab of your action tools from the control parameter and what your errors are. You need to use a batch macro (with two control parameters at least for multiple file types)...

Have you seen the weekly challenge on this? maybe look at the solutions there? https://community.alteryx.com/t5/Weekly-Challenge/Challenge-180-Thousand-File-Challenge/td-p/461356