Macro to iterate through files in a directory

I have a series of Hierarchal XML files in a directory. When I try to process these files using the Directory and Dynamic Input tool, I am running out of memory - I have this as an open issue with Alteryx - this method is off the table for now.

As a workaround, I would like to iterate through each file in the directory, processing them one at a time.

I am a experienced developer, but new to Alteryx.

I would like to have a source workflow that creates a list of files. I then want it to read each file and pass a data stream to a macro file for processing. That macro will then pass back a result stream. The base function can then do final processiing on this.

Can anyone provide me some high level strategies for performing this? My first question is how to iterate through each file?

Thanks,

Marc.

Developer Tools

Batch Macro

Accepted answers

JordanB

Hi @nbt1032

Correct each file is processed individually in a batch macro. Once it finishes one record it then processes the next.

All the parameters will come from outside the batch macro (directory tool). In this case we are updating the input file path with those coming from the directory tool.

In the action tool we have chosen 'update value' this allows us to update part of the tool it is attached too. You can add more interface tools (this is a category) and paramatize other parts of the workflow if you want to as well.

You do not have to have a macro output.

I would recommend filtering this file when you read them in from the directory tool. This way you can deal with that file seperately.

Best,

Jordan Barker

All comments

JordanB

Hi @nbt1032

I have attached a module (built in alteryx 10.6) which iterates through 13 XML files. You can open up the macro and insert your own workflow.

Please note the input file within the macro reads the file as a non-delimeted CSV and the rebuilds the XML and parses it out using our summarize and xml parse tool. This may help with your memory issue.

Best,

Jordan Barker

Solutions Consultant

XML Sample Workflow.yxzp

nbt1032

Jordan:

I have a couple of questions about the workflow.

There is no "explicit" looping of each file in the specified directory. Is each file processed indvidually in this logic?

TIn the batch macro function, the first object receives the full path of the file to process. The 2nd object is an Action. Where are all of its parameters coming from and how can I update them?

In this example, all files have been concatenated, which is what I wanted. But, what if I did not want any output from the batch macro. Is this required.

Finally, I want a special action for the first file. How would you recommend handling this.

Thanks for the sample. It is very helpful.

crookie74

Hi. I'm new to Alteryx so sorry if this seems a silly question. Can you tell me how you added the call from the "XML Sample Workflow" workflow to the "XML Parse Macro". From my googling people seem to use the runner macro, but I tried to recreate your "XML Sample Workflow" using the runner, but the xml data didn't come out in the final results.

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703