Unable to access API with download tool

[JAS7703]

I'm trying to extract data on Sainsburys sites.

https://stores.sainsburys.co.uk/api/v1/stores/?fields=slfe-list-2.21&api_client_id=slfe&lat=54.09407...

If you go to this page, you can see the json which I want to download. Ideally I would loop through all the pages but I get an error message when I try and access with a download tool in Alteryx:

{"errors":[{"code":23103,"detail":"String expected","id":"ca517267-7fb0-4458-a65a-1df2a3ebaecf","meta":{"value":["by_distance","by_distance"]},"source":{"parameter":"sort_by","pointer":"/sort_by"},"status":400,"title":"Bad Field Value Type"}]}

Anyone have any ideas?

[BrandonB · Accepted solution]

You need to leverage an iterative macro to deal with the offset. Other APIs call this pagination which basically means that only a certain number of records are pulled at one time. I have included a screenshot of the macro below along with a workflow that does it for you. 

 

[JAS7703]

Hi Brandon,

Having looked through your solution it seems the error I was experiencing was caused by the "Encode URL Text" checkbox. I had this checked, whereas you had it unchecked.

I had never really noticed this checkbox before but thinking back, it has probably been the cause of a few issues in the past!

Thanks for helping me realise what was going wrong! 

Josh

[BrandonB]

Yep most of the time the Encode URL Text option is a good thing because "When checked, the specified URL will be encoded as needed where unsafe ASCII characters are converted into a format that can be transmitted over the internet. An example of this would be the substitution of %20 for a space." In your situation it was probably converting a character into something that was actually different from what you had intended to pass through with the URL:

https://help.alteryx.com/2020.2/Download.htm