API pagination - Headers

[harinder1301]

Hi,

I have API URL which uses Bearer Authentication and is working ok in both Postman and Alteryx workflow. But i can retrieve data for only page size up to 1000. I want to create iterative macro.

But, I cannot see the header information containing TotalPages and total count in JSON downloaded data column. But, I can see this total pages information in GET response header using postman tool.

Please see attached screenshots.

Appreciate your advise on retrieving this total pages and total count information in Alteryx.

Best Regards,

Harinder

As per my API documentation , 

• Requests that return multiple items will be paginated to 100 items by default, but you can change this size specifying the pageSize parameter, up to 1000 items.

  You can specify further pages with the pageNumber parameter, the first page will be the number ‘0’ (zero).

  The responses with paginated items will return the paging information in the header, as is it shows in this example -

  Content-Length →31200

  Content-Type →application/json; charset=utf-8

  Date →Thu, 14 Jul 2016 09:10:01 GMT

  Server →Microsoft-IIS/10.0

  X-Pagination →{"TotalCount":73,"TotalPages":2,"PrevPageLink":"","NextPageLink":" URL...."}

  X-Powered-By →ASP.NET

  The X-Pagination header will expose this information:

  • TotalCount: number of total items that matches the filters.
  • TotalPages: number of pages for all the items.
  • PrevPageLink: a link to get the previous page of items, if it is the first page, this will be empty.
  • NextPageLink: a link to get the next page of items, if it is the last page, this will be empty.

[danilang · Accepted solution]

Hi @harinder1301 

The response headers are returned in the DownloadHeaders field and not in DownloadData

Dan

[harinder1301]

Great, Thank you! I had checked this before but it was showing me HTTP 200 OK . I have copied and pasted to notebook.  

I would appreciate if you can please advise me the tool to format this output:

access_token Get url Authorization DownloadData DownloadHeaders
YfAvlLT4w32XEVVVtIfbuIK7GtMx7y-GOFfoqRKyeemZvTVIuRwebk3omeZAV1YSIc2... https://01...&pageSize=1000 Bearer YfAvlLT4w32XEVV. [{"sNumber":"123","sName":"B","cCode":"LE" HTTP/1.1 200 OK
Date: Thu, 08 Apr 2021 14:56:50 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=d9f0aa726072db684d33e0c5717e96e5c1617893810; expires=Sat, 08-May-21 14:56:50 GM...

Thanks

Harinder

[danilang · Accepted solution]

Hi @harinder1301 

Use a Text to Columns tools configured to split to rows with \n as the delimiter.  The pagination info that your looking for will be in one of the rows.  It will have the same format as the equivalent row in your postman header output, so use that format in a filter tool, either Startswith() or Contains(), to extract the row you want.

Dan