Looping API Call Until Results are NULL

[WindDoh]

I'm using swapi.co to pull a list of Star Wars data. The API returns 10 results per call and provides a URL for the next 10 in the list. If I was trying to pull the character list, I would need to copy my workflow 9 times to get all 87 results using the "Next URL" from each call. I'm attaching the workflow I created, which copies the data 9 times.

How do I write a workflow that would loop through until the results are [NULL]. I feel like the answer has to do with an Iterative Macro but I cannot figure out how to do it.

My use case at work has an unknown number of results and would be closer to 16,000 results per quarter. The API I'm using gives 100 results per call but that's 160 times I would need to copy the workflow just for one quarter's worth of data.

HELP!

[JosephSerpis · Accepted solution]

Hi @WindDoh I have actually built a workflow to extract data from that API. I've attached my workflow to this post. Essentially you want to use an Iterative macro to loop through and select the next page then extract the results. The iterative macro I used to solve this for this API is included in the workflow.

An alternative is to use generate row tools to produce the number of rows required (e.g. in your case 160 rows) with the API call for each page you want to extract data.

This post may be helpful if you want to use the generate row tools approach.

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Limited-number-of-records-can-be-scrap...

[kimloh]

swapi has changed their API, and i tested the new url with your macro & workflow, works like a charm!. Definitely helped me to understand macro & iteration function better.