Hi everyone,
So I have a we API that I am calling to first get all the historic data out. It limits me to 10000 rows at a time and i believe there's about 200000 rows of history.
It's a typical url with input parameters. I start off a "dateNum" parameter at 0 on the first call and then repeatedly call it using the max latest figure returned. Unfortunately it doesn't go up in single integer step but you always use the max number returned in the next call.
I was manually inputting the new "dateNum" figure each time and running my workflow to append the new data to a table in SQL however this was getting quite messy .
I would like to find a way to run my work flow but have the new "dateNum" value selected based on the max number in the datenum column returned and then dynamically use that to run again etc until there's no more data to get out.
I thought I could write my end table to sql db and then have an offshoot to select the new parameters and re-run the flow. However I am not sure how to connect this all up and set it running with a log of how many calls were made - and if this is possible?
I have attached my current workflow (with my api credentials not in it so it wont run) just to show the process .
Any help will be much appreciated!
Thanks
