Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

Run a workflow on multiple files and output the result separately for each input file

rohan_tibarewala
9 - Comet

I need to run the same workflow on multiple files saved in a folder and write the output separate for each of those input files.

I have found a batch macro which can read all the files and run the workflow on it, but the issue is that it is producing a single output with the results of all the input files unioned together.

Please advise a solution !

14 REPLIES 14
NicoleJohnson
ACE Emeritus
ACE Emeritus

Without seeing the macro/workflow, this is just a guess, but you can usually specify if you want to bring in the filename when you input your multiple files to the workflow, and then you can use those as a grouping option when you go to export at the end of the workflow. Depending on which output method you're using, you should be able to append a filename or a field that will differentiate your file names onto the end of each file, which will create those separate files you're looking for...for example, the section at the bottom of the Output tools gives you the option of "Take File/Table Name from Field," where you can then identify if you want to append/prepend a field to create the file name, or replace the whole thing, etc.

 

Does that point you in the right direction? :)

 

NJ

rohan_tibarewala
9 - Comet

Thank you so much Nicole. 

Your hints helped me a great deal and i am very close to getting the results as i want from the workflow.

Like you mentioned, i was able to add a field with filename using "Output filename as a field" in Input data tool and also the outputs are grouped using Output data tool, based on those file names perfectly.

However, I have hit another bump for now and hoping more of your help.

In the final output i have noticed that the formulas/tools in the workflow seems to have run after taking the entire data together from all input files and then just the output is split based on file name. I was hoping that the workflow would run on each input file individually and produce the output for each file. To achieve this i tried placing 'Block until done' tool at different stages of workflow but without any luck.

 

For ex. I need to calculate the sum of field 'A' for each input file (lets say it would come 100 for File 1 and 200 for File 2) and i want to see this result in output of both the files. But instead i am seeing the sum only on File 2 as 300.

 

Hope my query is clear.

NicoleJohnson
ACE Emeritus
ACE Emeritus

Absolutely! There is always the option of going the batch macro route and processing one file at a time (macros are not my particular strong suit, though there are several other macro wizards here on the Community who could point you in the right direction!)... however you might be able to achieve the same thing by making sure you group based on file name in any of your Summarize tools/etc., which would give you the sum total for each file rather than for the total across all files. I'd probably need to see a bit more of your workflow though to really troubleshoot... can you share a sample version of it with some dummy data, by chance?

 

NJ

rohan_tibarewala
9 - Comet

Please see the attached workflow and related batch macro.

rohan_tibarewala
9 - Comet

Also attaching the excel files that i used as inputs for this workflow.

rohan_tibarewala
9 - Comet

Thanks a million Nicholas.

Your hint ("making sure you group based on file name in any of your Summarize tools/etc.") was spot on. 

It worked for me !!

anamikakatiyar
5 - Atom

Hi Rohan,

 

I am attempting something similar to what you did. Can you please share your workflow that worked for you in order to add a particular column from multiple files placed in a folder/ directory.

 

Thanks,

Anamika

rohan_tibarewala
9 - Comet

Hi Anamika,

 

I created a workflow which includes couple of macros to accomplish what you are looking for. Please see the attached package and let me if there is some issue or if you have any question. This can be used to run for multiple files or even multiple tabs within the files as well. For getting the output also in separate files, you need to make slight changes in the Render tool or add Layout tool or use Output data tool to split your output based on file name column. I developed to simply create a temporary excel sheet containing different tables from different input files.

TarunDeep
8 - Asteroid

Hey Roahn,

 

Your macro works quiet well on the files. I have modified it a bit according to my requirement, but I have one issue, where the input folder should read all the files (i.e *.*).

As soon as I make it extension specific (*.csv or *.xlsx), it throws an error. Have you encountered something similar? How can I make it file type specific? Let me know.

 

Thanks for the Tool.

Labels