Download Tool only returns 100 items

Question

I am using a Download Tool to connect via API, and the source data should have about 3200+ email addresses that should be returned. But for somes reason, I am only getting 100 items returned.

In the output, the totalcount is correct (3248 relevant items), but as shown here I only get 100 of them.

The action type is "GET", and I have added parameters to maybe fix the problem, but they did not:

Content-Type - application/x-ndjson

accept - application/x-ndjson

I have also tried changing the output from String to Blob, but still only end up with exactly 100 items.

Googled a lot as well, no solutions found. Anyone in here who might have tried this as well, or have an idea for a fix?

gyang3 · Answer

@MartinJacobsen there is a high possibility that as @Felipe_Ribeir0  mentioned, the API you're querying from limits you to a max of 100 records. Thus, increasing the limit above 100 would not make a difference. In this case, you would need to use pagination to return the rest of the records. There are quite a few community articles that discusses pagination workflow within Alteryx. A few below may be helpful to get you started on the right track

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Iterative-macro-for-API-pagination-retrieval/td-p/34456

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Iterative-API-call-with-pagination-amp-updating-URL/td-p/450057

Hope this helps and please mark as solution if so!

MartinJacobsen · Answer

Thanks for the replies. I have also previously checked the (rather lacking) documentation, and tried the "limit=100000" parameter as well without success.
The paginated thing sounds like a thing, I have now made a ticket to the API provider and asked for details. At least now I now that it (probably) isn't an Alteryx issue...

apathetichell · Answer

I've also seen some APIs (Oracle FAH for example) with a default limit. You can overwrite this with something like Limit=10000 - so you may be able to overwrite this in your API call.