Using Throttle Tool in iterative macro

[fardeen9983]

I have an iterative macro that fetches data from a paginated API. The gateway has a strict limitation of 100 requests per minute.

Where should I place the throttle tool in my workflow to ensure that iterative macro runs so that the 100 requests/minute limit is held? Placing it inside the macro doesn't make any sense as for each iteration only one request is made.

Also to note here, the iterative macro itself is part of a batch macro.

P.S. Cant upload the workflow.

[AngelosPachis]

Hi @fardeen9983 ,

It's a tough one to call if we don't know exactly what's going on in the workflow, but as you mentioned, it would be meaningless to add the throttle tool inside the iterative macro.

What does the batch macro do? If it is replacing certain parts of the URL string, then maybe it would be a good idea to place the throttle tool inside the batch macro but before your iterative one; but that's a speculation as I'm not sure how many records pass into/through the batch macro streams, feeding to the iterative macro.

[pedrodrfaria · Accepted solution]

Hi @fardeen9983 

I believe you should do an additional macro based on the iterations. Create a new iterative macro that looks at every 100 digits difference (between 0 to 100, 100 to 200, 200 to 300) and in the Final output you add the throttle to wait X amount of seconds for you.

So you would have one iterative macro inside of one iterative macro that is inside the batch macro.

Let us know if you need help with this solution.

Pedro.

[DanielG]

I used the tile tool to create groupings that were much less than 100 records per POST to avoid that issue entirely.  (My data set will always be in the range of 8000-12000 so I can set tiles to easily be less than 100 records per, so if you data is more widely varied that might not be a good idea for you.)

The workflow feeds the groupings into a batch macro and I build the payload on the fly from the records from each tile.  As mentioned previously, not sure the throttle tool would have much impact on less than 100 records at a time.

If you do a GET and push 1 record through at a time, the throttle tool might be of use, but I am not 100% sure there as I have mainly been playing around with POST for my API.

Good luck and please let us know how it goes. 😀