Throttle an Iterative Macro

[MZ]

I have a working Iterative Macro for making HTTP GET requests using a data harvesting protocol.

Each run of the Iterative Macro outputs a new value for a Resumption Token which is passed back to the Iterative Macro input to allow the next iteration to pull data from where the previous request finished.

However, I need to apply additional throttling to comply with external limits.

I have tried putting the Throttle tool within the Iterative Macro (before the Download tool) but it does not affect the rate - I assume because the throttling rate for each iteration is calculated separately.

Is there a way of putting throttling on the Iterative Macro itself, or inserting a Throttle tool in between the Output and Input or the Iterative Macro?

[PhilippK · Accepted solution]

Hi @MZ ,

what external limits are you struggling with? What are the exact restrictions?

Maybe a "Wait a Second" tool after your "Download" tool already helps.

Best regards

Phil 

[MZ]

Thanks Phil,

I need to leave at least 10 seconds between each iteration, under the terms of my agreement with the provider.

Adding 10 seconds to each step would work as a brute force solution - thank you.  It does introduce an unnecessary delay for those cases where the iteration itself takes time (e.g. waiting for the service to respond), which isn't as ideal as throttling the whole macro would be, but it will certainly work.

Michael

[BretCarr]

Here is the way that I throttle my iterative macro when waiting for a batched process on the remote API side to finish so I'm not creating a DoS looking attack by pounding their server for a couple of minutes while my process waits for the good response: