I have a large file (500mb) that my workflow has to download and then process.
I noticed that the Download Tool downloads the file once, then overwrites the file and downloads it a second time. I watched the folder that the file is downloaded to and confirmed that this happens.
Is there a way to avoid this? Am I setting up my Download Tool incorrectly? Unfortunately I cannot share the link, but it is being imported from the URL field of a connected text input tool.
Solved! Go to Solution.
Possibly? That's kind of my question. The problem is it takes about 8 minutes to download the first time, then it spends another 8 minutes overwriting it and downloading it the second time. Ultimately it's not a massive problem but I was curious if I'm doing something wrong in my settings or not.
How many rows your download tool is receiving. Could you check if you're not sending the link twice?
Hi @abragg0103 ,
as @AZuc suggested you're probably sending the link multiple times to the download tool. If you are dynamically building your URL based on logic, or if you are scraping the sub-URLs from the parent page, then you will need to add a summary tool and group by the URL field. This will then produce the unique URLs that you are downloading.
M.