This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
2. When I go to the link output from the macro, which should match the URL, it redirects me to the first page, which means that for 2 different URLS going into the data harvest macro, only results from the original page are returned.
DM 2 has a few parsing errors to fix as well but works fine for this purpose.
I wasn't able to follow all your parses to see what data you were trying to get to, but you shouldn't need a macro to extract the data. Attached is a workflow that determines the number of pages in the link from the page counter at the bottom of the first page, then creates web addresses for each page. Data investigation indicates this website uses a different method of pagination other than page=1, page=2, etc., which this workflow addresses. Then all pages are fed at once into the Download tool and the data for all pages is in the DownloadData field. You should be able to break out the data by newlines then further parse into what you are looking for. Let the community know if you need further assistance parsing the data, but it looks like you are well on your way.
Without seeing how your workflow is setup, there are a few things that we need to look at.
1) Is your iterative INPUT setup correctly? if you don't have one of those setup, then there is nowhere for the iterative output to go back through the macro.
2) It looks like you have a formula where you are looking at engine.iterationNumber. whenever you do get the lack of iterating fixed, this engine.iteration number will start at 0. So you may want to think about some way to stop the first iteration from creating a URL for the second iteration back through the call for page 1. Essentially, like if engine.iterationNumber = 0 then 2 else engine.iterationnumber + 1
3) @T_Willins is correct, you don't have to do an iterative macro. Their solution works quite well.
Here is an article I wrote about iterative macros and how to set them up, there might be something here you haven't thought of through your process.