Macro for running iterations dynamically

[ravitejapaideti]

Hi All, I'm building a workflow which requires Alteryx to pull data from another vendor application (Archer) via API call. Created a report with required columns in Archer application and by default, it displays 50 rows/records. One have to scroll pages and the created report has 40k records approx. This might fluctuate dynamically based on processes happening in Archer application. Coming to Alteryx, API was able to ready only 1st page records (50). We changed the page number and it fetched the next 50. Is there any way to build macro in such a way it should run iterations to fetch all the records? I'll get the record count via API and /50 gives me # of pages the API need to process. Any inputs are highly appreciated.

[jonnyrask]

@ravitejapaideti - Interesting question. Here is what you could do.

Create three workflows: one a standard workflow, the other an iterative macro, and the other a standard macro.

The workflow starts in the standard workflow:

Call the API to get the count of pages that you need to run the other API on. Call the standard macro and pass in a row with the number of pages.

The standard macro should take in a text input with a generate rows tools that creates like 1000 rows so that there is a recordID for those rows from 1-1000, then using the number of pages from the API call in the standard workflow, the Sample tool should be used to only take the top X number of rows where X is the number of pages. The last part of this macro should call the iterative macro and pass in the table with the count of rows that go up to the total number of pages you want the API to run on.

The Iterative Macro should run as usual, but pass in the current row value to the API so that the correct page can be called.

With some more thought a better solution could be used, but off the top of my head, this should work.

[danilang]

Hi @ravitejapaideti 

If you can find the number of pages before getting the data, you might be able to do this without a macro.  

Just build the list of URLs and pass them into a download tool.  Set the Maximum Connections to 1 in the Connection tab of the Download tool config.

[TimN]

@ravitejapaideti Could you share any details on the API call to Archer?  I'd like to try this.  

Thank You,

Tim