Download file wont extract URL's inputed in text file

[zaina1498]

Hi, I'm unsure why the download tool won't download the URL's properly and extract them and output them. Is there anything I can do to fix this? Why might this issue be occurring? I've attached the workflow below, any help would be greatly appreciated!

[atcodedog05]

Hi @zaina1498 

I tried you workflow. It seems to work.

What exactly is the issue?

[TheOC]

Hi @zaina1498 

I had a look at it looked like some of your request headers were coming back as wrong.

I have added a user-agent to the headers field of the request, and this seems to have fixed the issue on my side. Does it fix it for you too?

Attached below is the workflow

[atcodedog05]

Hi @TheOC ,

Thats powerful tool you have got there can you explain more about adding a user-agent to the headers field of the request

[TheOC]

hi @atcodedog05 

Yeah ofcourse - As far as I understand certain webpages try to stop scrapers by requiring certain information/formats of the request. By adding this header, from what I understand it is telling the site that it is requesting from, that the request is coming from a webpage. This gets past a lot of 'firewalls', as many sites will try to stop requests from python scripts for example. 

So as far as the site is concerned, someone has viewed the page from a browser, and so delivers the information.

Hope this helps!

[zaina1498]

Hi, @TheOC  thank you for getting back to me! However it still wouldn't run for me. It stops after a while of it running and gives me an error when I run the workflow you attached. 

[TheOC]

very interesting, my workflow works with no problem at all, i'm curious as to why it wouldn't for yourself!

[zaina1498]

Do you think it could be a difference in Alteryx settings or versions?

[atcodedog05]

Hi @TheOC 

You are speaking about this part right. Neat

Its working for me too.

[TheOC]

Hi @zaina1498 ,
Very potentially, What version are you on with Alteryx? I'm not quite on the newest so ill try updating to your version and seeing if I get the same issue!