Download tool and JSON Parse tools resulting in "too many requests" error message from API

Good morning Community,

I have an API from a vendor that has a 5-second rate limit. I am using the Download and JSON Parse tools to download the data, but I randomly get an error from the API that says "too many requests," meaning that I am calling on the API more than one time in that 5-second window. Sometimes my WF is great and downloads all of the data, and sometimes I get this error and I can't add the new data to my table. On the days I get this error, I double-check my URL is correct by pasting it into Chrome and all of the data is there. This makes me believe my tools are causing this. Has anybody dealt with this before?

Developer Tools

API

Download

Accepted answers

asieveking

I had to develop the function that performs the same action as the "Download" tool attached in the sample file.

I hope it is useful to you, greetings!

Rate limit with Python.yxmc

All comments

Treyson

Hey @Shelbey,

How any input records are going into the download tool? Is this happening when you only pass a single record? Can you disclose the vendor?

Shelbey

Hi @Treyson

I have one text input tool that has only one record for the columns URL and KEY. The output of my download tool looks like this:

And the output of my JSON Parse tool looks like this:

I have absolutely no idea what I can and cannot disclose here, but I don't believe this will give anything confidential up. Hopefully there is enough information there for you. I can also provide screenshots of how the tools are setup if needed.

Bigum

Hello @Shelbey - I'm encountering these issues, and was wondering if you found a solution?

Shelbey

Hi @Bigum

In this thread, there is a file attachment someone else sent me a while ago that uses Python tools. I've been using that process ever since it was sent to me and I haven't had any issues with the "too many requests" error ever since.

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703