looping through dataset (for loop?)

[Pav]

I've a dataset that is variable in size. I want to loop through it and have an output unioned on one excel sheet.

The dataset may vary between 17-204 "groups" each is uniquely identited as I assign a recordID. (I first group them using summarize tool)

Each group also has a variable number of rows (between 9-14) but that isn't an issue as I created a macro that is able to process this and create a desired output.

My problem is that currently I can only do this for 1 "group". I looked at batch and iterative macros but I'm not able to come up with a solution.

What I want to do is to loop through first group, apply my macro, then loop through second group apply macro again and union the results. Continue this process until the last group. 

My thoughts were around for loop as I can easily specify the limit of iteration (count records of groups) but I struggle to develop a solution.

If any more info is required, please let me know. I appreciate any help 

I attached a sample workflow. At that stage I would like to apply my macro to run through #1 group, then loop through and run through #2 group and union the results with those from the first group and finally run the macro on #3 group and union those results with the previous two to create a final output. 

Thanks

[pcatterson]

This is exactly what a Batch Macro would do.  Here is a simple example of a working Batch Macro and its application in a workflow that hopefully will give you the idea of how to use it.

[JordanB · Accepted solution]

Hi @Pav

I am not sure if I completely understood your use case however I have built a workflow and macro which might help guide you. This macro will feed in one sample group at a time (Using a control paramater interface tool), so therefore if I understand your use case correctly, you can just add your current macro workflow within the one I have, which will pass one sample group at a time, process your workflow and then union the sample data as an output. 

Workflow built in Alteryx 10.6

Best,

Jordan Barker

Solutions Consultant. 

[Pav]

Thank you both.

@JordanB

Your solution works perfectly, thank you! I just added my macro after the join in your batch macro and everything works as I wanted.

Could you explain how that batch macro works? I would like to understand every step and right now I struggle to understand how the RecordID is incremented (when) and how the output is unioned as well as how the macro knows when to stop. Sorry for all these questions, but I would like to reuse this in the future but to do that I want to know exactly what is happening at each step. 

Thank you very much. 

[JordanB]

Hi @Pav

No problem at all! A Batch Macro is one of the more complex topics within the designer but as you can see the power is significant. Here is my best attempt/analogy to describe this via text:

On a high level a Batch Macro processes one record at a time, similar to a intravenous drip at a hospital. The intravenous drip has a gauge on it which passes one drip at a time into your body. This same process happens within a batch macro but the 'guage' is the control parameter and the intravenous fluid is the list of data values you pass into the control parameter anchor (Highlighted in red below).

Because we attached the control parameter to the filter in the macro, it will now only process, one record at a time, the values you pass into. Therefore in your example, it will process recordID 1, output those results, it will then reset, pass through the record ID 2 value updating the filter tool and then only pass through that value through the filter and union the associated data to the first dataset passed through. This process continues untill all the values has been fed into the control parameter. 

Often people will use a batch macro to read in multiple excel or text files at one time. They will actually have a control parameter updating an input tool file path. Then from outside the macro will use a directory tool to feed in the file paths from a folder and process then all at one time and union them together. 

Here is a good article explaining the above example. 

I apologize for the obscure anology but this often helps people visualize the process. 

Best,

Jordan Barker

Solutions Consultant. 

[Pav]

This is great!

Thank you Jordan, that explanation makes perfect sense and now I know exactly how this process works, thank you for that.

[Greenlama254]

Hi JordanB

Thanks for explaining how a batch Macro works.. It is still kind of sketchy to me to be honest..

You mentioned in your explanation that "and union the associated data to the first dataset passed through". However, what if you do not want to union the data but you want the batch macro to process the data 1 line at a time to generate a report per line? I just asked a question about this in the community, but I just stumbled upon your explanation here and your reply comes closest to what I am aiming for. 

Generating a report per line of input - Alteryx Community

Hope you can help, thanks

Roger