nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Method for paging REST API results

GreggD

I have over 30,000 records I need to retrieve via my Rest API call, however my REST API limits the number of records we can return to 1,000 records. SO I need create a paging solution. In a regular procedural language I would use control flow logic like a WHILE loop to control pulling 900 records at a time, until I reach the last record and then stop. Is there a way to accomplish the same thing in Alteryx? Thanks in advance!

API

Accepted answers

GreggD

Ok - after a lot of searching I finally found the following video in the Alteryx Academy that helped me to at least get the basics down:

https://community.alteryx.com/t5/Videos/Build-Your-First-Iterative-Macro/m-p/55565

All comments

DiganP

@GreggD This article posted earlier talks about this, please take a look!

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Iterative-API-call-with-pagination-amp-updating-URL/m-p/450644#M87033

You would have to use the iteration macro. You have to use the Interface Designer to set the loop. Part of the look will be the results and the 2nd part would be to create a string of the next url. Attached is a sample workflow that I like to show to clients around this use case.

Web Scraiping with Pagination (1).yxzp

GreggD

Thanks for the reply. I am using version 2018.4 of Alteryx and am not able to load your workflow since it is a newer version. I also could not find the Iteration Macro in Alteryx Designer or in the Gallery. Can you point me to where I can find it? Thanks.

GreggD

Thanks Cady for your help. I was able successfully load the workflow after following your instructions. Digan, When I loaded the workflow, it said that I was missing macros. Once loaded, the workflow only had a Text Input tool and a ScreenScraper macro with no connecting lines. Was there supposed to be another macro? You had mentioned that I should use the Iteration Macro, however I have not been able to find that macro anywhere including in the Alteryx Gallery. So I am still stuck. Any guidance, such as where I can find the Iteration Macro, would be appreciated. Thanks, Gregg

GreggD

Attached is my workflow.
I am hoping someone can see what I am doing wrong.

There appears to be a correlation between my workflow and the Interface Designer, however I don't know what it is. Is there anyone who can help me connect the dots. I just need to understand what Interface Designer is expecting so I can give it what it is looking for.

Workflow.png

Quick Links

This months top contributors

atcodedog05 19598

Qiu 15905

binu_acs 15783

MarqueeCrew 13709

apathetichell 13703