Is there any way to split data for a batch macro?

Hi there, I have a batch macro that I have some calculations that I do inside of it. It has lots of formulas and it is working pretty rough when I try it with a big data (more than 1 mil row). It sometimes works sometimes gives an internal deadlock error. I tried to optimize my macro to it's finest and it still is the same.

I'd like to use the same macro of mine but if I can I'd like to have first 50 or 100k rows to work within my macro and afterwards the next 50/100k rows and so on until the whole big data set works. How can I achieve it?

Workflow

Datasets

Tips and Tricks

Macros

Batch Macro

Iterative Macro

Accepted answers

JamesCharnley

Hi @gorkemakinci

Splitting data up is, at the highest level, the point of a batch macro, so this is definitely possible. What you need to do is just create some kind of grouping field for every X rows of your data, and send that into your control parameter to group it. Let's say you just add a RecordID , and then a multi-row formula that says if the Record ID is a multiple of 50000, then +1 to this grouping otherwise leaving it as it is. And that should batch the desired number of records in every time the macro runs.

All comments

JamesCharnley

Hi @gorkemakinci

gorkemakinci

I totally got the logic and I added a record ID tool, however can you guide me with what would the formula look like for the grouping field? I'm assuming we want it to start from 1 and then after 50001st row it switches to 2 and so on, And that grouping field is going to be my control parameter for my calculator macro.

If I got that right I only need to solve how to formulate it in the multi row formula tool! Thanks for clarifying my inquiry about Alteryx Designer, wasn't completely sure if there was a specific way or tool to do this, and if you'd like to help more with the formula I'd be glad (whenever so I can compare what I've written for a better understanding on how other people work and code in Alteryx!) Have a nice one!

Quick Links

This months top contributors

mceleavey 383

mbarone 337

Hollingsworth 335

LanisC 335

RithiS 280