Iterative Macro with API pagination

[brwright]

Hello, I have an API that I pass a limit and an offset.The first pass I have a constructed URL string that starts at offset=0

The first pass returns a JSON Value for a property called "nextPageLink" which already has the next offset.

From my workflow, I get an authorization bearer token and pass that to the macro call.

Inside the macro, the first tool I have is a formula that sets the initial URL value. 

I am getting the results and splitting everything except the "nextPageLink" to the output. I send "nextPageLink" to the Loop. 

On iteration 1, it works fine. On Iteration 2 I am getting error:  RPSMacro (63) Iteration #2: Tool #31: The field schema for the output "Loop" changed between iterations.

I am so stuck my eyes are crossed. 

[gabrielvilella]

To get rid of the schema error you need to just select the Auto Configure by Name within Interface Designer > Properties area.

[brwright]

Thanks, I had done that but I did it from the standard workflow instead of the macro workflow. (This wasnt clear in the help article I initially read). That schema message has gone away. Now I have the following I'm staring at with zero percent.  

I set the macro's max iterations to 2 so it would stop and I could take a look at whats being returned. Seems to be in an infinite loop regardless of that setting.   If I stop it, I cant see any output. 

[brwright]

2 iterations ran. I have the max iteration at 2. I dont understand why it's still going. 

[gabrielvilella]

8537 iterative records remaining is just the record count for the iterative output anchor. It means that you still have records going through the iterative output. The iterative macro will only stop when there are no more records left on the iterative output or the maximum number of iterations is reached. 

[gabrielvilella]

Are you sure you are sending just the nextPageLink on the iterative output? I believe this should be just one record, and when there is no next page there are zero records and the macro stops.

[brwright]

I have to send an Authorization token and the nextpageLink. How do I validate "what" is getting sent each round? I dont understand the relationship between the loop and where it replaces the url and gives the download the Authorization header payload.

[gabrielvilella · Accepted solution]

The data that is being returned to the input is the one you have defined as the iteration output within interface designer.

[brwright]

The Input only has Authorization. So I thought maybe I need to include a URL too. The 2nd run was "unauthorized" . Now I'm getting new error that URL was missing. What a battle.

[brwright]

Got it working, I shuffled things around several times until it was "happy". I have my 512k results now!