JSON API Data Download Limitations

I am trying to pull some data down through an API but I seem to be limited to 100 records of data (JSON Name 3) at a time. I believe I need to run an interative macro but I do not have the number of pages only number of total records. Does anyone know how I can get the full data set (or 883 records)?

Macros

API

Iterative Macro

Accepted answers

All comments

Christine_Assaad

Hi @pbaba

Please check this post: https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Loop-through-API-calls-until-all-data-is-downloaded-Iterative/td-p/519435

I believe it's related to pagination which could be handled using a macro as described above. Hope this helps. Cheers!

pbaba

The issue I have is that I do not have a way of knowing how many pages there are. There is a Cursor field but I am not sure if it plays a role with pagination.

danilang

Hi @pbaba

The methods for handling pagination are API specific. Sometimes there's a link to the next data chunk in the DownloadHeaders field. Sometimes there is an offset parameter that you send in the URL. Look into documentation of the specific API that you're calling.

Dan

pbaba

@danilang Thank you very much for your insight. It is appreciated.

Quick Links

This months top contributors

AkimasaKajitani 389

mceleavey 388

mbarone 337

michael_weaver 335

Hollingsworth 335