Stopping and Separating Throttle records on a condition

[fardeen9983]

I have a Throttle tool passing records at the rate of 100 records per minute to a download tool which uses an Auth token expiring in 10 mins. If the number of download records to be called exceeds 1000 then the time taken will more than Token lifespan, resulting in 401 error status. This will be unnecessarily lead to n number of calls with invalid token 

I want to add this condition so the download tool stops accepting records once 401 status occurs and in the output has 2 streams - One for non 401 responses and  all others which will need a new token

P.S. I do not want to create a separate macro for this if possible

[danilang]

Hi @fardeen9983 

To do this with no changes to the inner macro, I would suggest taking your existing macro and putting it in an outer batch macro.  In the main workflow determine the total number you need to download and divide this into batches that will take less than 10 minutes to execute.  Pass all the records and the batch ids to outer macro.  In the outer macro, get your auth token at the start of each batch and send this and the current batch of records to the inner download macro.  Once the inner one finishes, the outer one will request a new auth token for the next batch.

If you're willing to change the inner one, convert it to an iterative one that contains the logic that gets the new auth token when required

Dan