Iterate Download tool based on the output

Hi All,

I have to make a HTTP request using the Download tool to get 999 records in Json format. Along with the JSON output there will a Nextlink (another Http link) to get the next 999 records and the process goes on untill the Nextlink is empty. So I have to make a iteration at Download tool based on the output data.

How can I achieve this? Anyone faced similar situation?

Iterative Macro

Accepted answers

Jonathan-Sherman

So you will need to build an iterative macro to solve this. Create your standard workflow (with a macro input) and filter at the end to separate out "next link" and "JSON data". One macro output tool after each and in the interface designer select the macro input, iterative output will be your macro output from output anchor giving you the "next link". You will need to ensure you have selected iterative macro in the workflow options tab of the configuration pane to give you the correct options in the interface designer. This will "loop" the workflow and use the output of the "next link" stream as the input once again, through the download tool etc and finish running when no further data comes out of your iterative macro output "next link". Assuming you don't hit the maximum number of iterations defined in the interface designer, which can be changed.

If this solves your issue please mark the answer as correct, let me know if you need me to build an example!

Jonathan

All comments

Jonathan-Sherman

If this solves your issue please mark the answer as correct, let me know if you need me to build an example!

Jonathan

afv2688

Hello @KarteekChalagondla,

It is possible to perform what you are looking for, you just need to create an iterative macro for that.

Remember that when creating iterative macros, iterative input should be the same as the iterative output.

Regards

KarteekChalagondla

Hi @Jonathan-Sherman

Thank you very much for the solution. It works. But what If I set iteration limit to 100000 and the NextLink is blank at may be 250 or 300. Will it continue to loop, or throws exception?

KarteekChalagondla

Hello @afv2688 ,

Thank you very much. It helps alot. My first few trails failed due to the wrong assignements.

Quick Links

This months top contributors

AkimasaKajitani 389

mceleavey 388

mbarone 337

Hollingsworth 335

LanisC 335