I have created an iterative macro that fetches data over an Odata API that is paginated. So after each request I have to get the next link and fetch remaining records and so on.
There are more than 1M records and there are some limitations set by the API gateway.
1. Max 100 requests/minute
2. Response payload size < 10MB
Now I want to dynamically change my page size and slow down/amp up the iterations so that I follow both the rules.
So my questions are
1. Is there any way to calculate the response size from the Download tool?
2. Is there a way to make the iteration halt when it hits the threshold of 100 requests/min and continue when it's a new interval?
Thank you for your time in helping my query.
PS. Can't upload the workflow
Solved! Go to Solution.
Hi @fardeen9983,
In order to comply with API constraints:
I do get the count of all records, but the requested logic from me is to keep on running the iterative macro like a True while loop until there are no pagination link in the response.
i will look into the throttle tool.
Thanks for the answer. Much appreciated
Hi Fardeen,
I am also facing the exact problem. Have you got the resolution for the same then kindly provide.
I am feteching Odata API ( Successfactor API) and it is getting stopped at 51 iterations though it should run for 76 iterations.
Please help if you may.
Regards,