nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Throttle an Iterative Macro

I have a working Iterative Macro for making HTTP GET requests using a data harvesting protocol.

Each run of the Iterative Macro outputs a new value for a Resumption Token which is passed back to the Iterative Macro input to allow the next iteration to pull data from where the previous request finished.

However, I need to apply additional throttling to comply with external limits.

I have tried putting the Throttle tool within the Iterative Macro (before the Download tool) but it does not affect the rate - I assume because the throttling rate for each iteration is calculated separately.

Is there a way of putting throttling on the Iterative Macro itself, or inserting a Throttle tool in between the Output and Input or the Iterative Macro?

Download

Iterative Macro

Accepted answers

PhilippK

Hi @MZ ,

what external limits are you struggling with? What are the exact restrictions?

Maybe a "Wait a Second" tool after your "Download" tool already helps.

Best regards

Phil

All comments

PhilippK

Hi @MZ ,

what external limits are you struggling with? What are the exact restrictions?

Maybe a "Wait a Second" tool after your "Download" tool already helps.

Best regards

Phil

Thanks Phil,

I need to leave at least 10 seconds between each iteration, under the terms of my agreement with the provider.

Adding 10 seconds to each step would work as a brute force solution - thank you. It does introduce an unnecessary delay for those cases where the iteration itself takes time (e.g. waiting for the service to respond), which isn't as ideal as throttling the whole macro would be, but it will certainly work.

Michael

BretCarr

Here is the way that I throttle my iterative macro when waiting for a batched process on the remote API side to finish so I'm not creating a DoS looking attack by pounding their server for a couple of minutes while my process waits for the good response:

Quick Links

Popular Tags

This months top contributors

atcodedog05 19598

Qiu 15912

binu_acs 15783

MarqueeCrew 13710

apathetichell 13703