Advent of Code is back! Unwrap daily challenges to sharpen your Alteryx skills and earn badges along the way! Learn more now.

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

Using Throttle Tool in iterative macro

fardeen9983
8 - Asteroid

I have an iterative macro that fetches data from a paginated API. The gateway has a strict limitation of 100 requests per minute.

 

Where should I place the throttle tool in my workflow to ensure that iterative macro runs so that the 100 requests/minute limit is held? Placing it inside the macro doesn't make any sense as for each iteration only one request is made.

 

Also to note here, the iterative macro itself is part of a batch macro.

 

P.S. Cant upload the workflow.

3 REPLIES 3
AngelosPachis
16 - Nebula

Hi @fardeen9983 ,

 

It's a tough one to call if we don't know exactly what's going on in the workflow, but as you mentioned, it would be meaningless to add the throttle tool inside the iterative macro.

 

What does the batch macro do? If it is replacing certain parts of the URL string, then maybe it would be a good idea to place the throttle tool inside the batch macro but before your iterative one; but that's a speculation as I'm not sure how many records pass into/through the batch macro streams, feeding to the iterative macro.

 

 

 

pedrodrfaria
13 - Pulsar

Hi @fardeen9983 

 

I believe you should do an additional macro based on the iterations. Create a new iterative macro that looks at every 100 digits difference (between 0 to 100, 100 to 200, 200 to 300) and in the Final output you add the throttle to wait X amount of seconds for you.

 

So you would have one iterative macro inside of one iterative macro that is inside the batch macro.

 

Let us know if you need help with this solution.

 

Pedro.

DanielG
12 - Quasar

I used the tile tool to create groupings that were much less than 100 records per POST to avoid that issue entirely.  (My data set will always be in the range of 8000-12000 so I can set tiles to easily be less than 100 records per, so if you data is more widely varied that might not be a good idea for you.)

 

The workflow feeds the groupings into a batch macro and I build the payload on the fly from the records from each tile.  As mentioned previously, not sure the throttle tool would have much impact on less than 100 records at a time.

 

If you do a GET and push 1 record through at a time, the throttle tool might be of use, but I am not 100% sure there as I have mainly been playing around with POST for my API.

 

Good luck and please let us know how it goes. 😀

Labels