Tool Mastery | Download

[MattD]

This article is part of the Tool Mastery Series, a compilation of Knowledge Base contributions to introduce diverse working examples for Designer Tools. Here we’ll delve into uses of the Download Tool on our way to mastering the Alteryx Designer:

To do your best data blending, it is a critical need to have the flexibility to connect to as many data stores as possible. No puzzle reveals a complete picture without all the pieces in place, and the same adage holds true in analytics. While we’re proud to boast a list of supported input file formats and data platforms that may even be large enough for database storage itself, unfortunately, in the ever expanding world of data you just can’t catch them all. Enter the Download Tool. In addition to FTP access, this tool can web scrape or transfer data via API (check your data source – there’s almost always an API!), giving you access to even the most secluded data stores. With the examples compiled below, and the wealth of data accessible on the web, you can turn nearly any analytical puzzle into the Mona Lisa:

FTP Upload
FTP Download
FTP Automation
Connect to REST APIs – without coding! (Part 2: Authentication)
Web Scraping

By now, you should have expert-level proficiency with the Download Tool! If you can think of a use case we left out, feel free to use the comments section below! Consider yourself a Tool Master already? Let us know at community@alteryx.com if you’d like your creative tool uses to be featured in the Tool Mastery Series.

Stay tuned with our latest posts every Tool Tuesday by following Alteryx on Twitter! If you want to master all the Designer tools, consider subscribing for email notifications.

Additional Information

Click on the corresponding language link below to access this article in another language -

• Portuguese
• Spanish
• French
• German
• Japanese

[yashd]

Can download take input other than text input tool ? , if yes, could you please name it.  ?

Thank you in advance

[MattD]

Hey @yashd!

Yes - in fact, all tools with a standard output connection should connect to the download tool. The only change you'd have is mapping those fields to download tool configuration options.

Best of luck!

MattD

[AD2]

I have a list of urls and I need to figure out which of them are active. I used the download tool, but the tool stops when an erroneous/non-active url gets pinged (ex unable to resolve host name). I need the tool to not stop and process the entire list. How would I achieve my requirement?

[ywang]

Hi,

I have a question about the Download Tool.

I have a simple workflow of 3 tools - a Text Input, a Download and a Output. The workflow used to work well on monthly basis but recently got error with nothing being changed.

-In Text Input, just use the following url:

https://api.appsee.com/analytics/daily?apikey=1618615cf7604633a84d3b7d5c5e7a08&apisecret=21b8f5354bb...

-In Download Tool, select URL as the filed and default for all others.

For some reasons, the Output shows no valid DownloadData and DownloadHeaders.

If cut and paste the url in a browser, there're data available and it's supposed to be able to download.

Any idea what is going wrong here?

[russellclaude]

I have a problem with the download Connector Tool.

I am doing a very simple scrape of Facebook pages. My first approach to scraping using the Connector Tool worked perfectly, but it was not dynamic. I had loaded in 100 URLs with a Test Input tile, and I got exactly what I was after.

However, I want to use data from an external database to feed the URL list. When I pipe data from am Input Data Tool and some Filters/Joins/Selects, I am getting "400 Bad request" errors from facebook:

<h1 id="sorry">Sorry, something went wrong.</h1>
      <p id="promise">
        We're working on it and we'll get it fixed as soon as we can.
      </p>

I am even hard-coding the URLs from the new flow, and the URLs that work from the Text Input do not work with the Data Input. It seems as though Alteryx is not sending the correct headers when I am not using the Text Input tile.

[JP2]

Hello,

I have tried a few different times and not getting very far so I am trying here.  I have used the download tool to do some complex API stuff so pretty familiar with it but when I try to connect to this url (currently being done through excel) I am not getting the data back.  http://www.eichfarms.com/

Thank you very much,

JP

[RodLight]

@JP2

This looks like it's an exercise in "web scraping" where the Download tool is basically downloading the raw HTML contents of the page you are hitting from the URL. It is all pure HTML code that is the result and just needs to be parsed out (and when I say "just", that doesn't mean it's that simple). 

Although it may not look like it from results of the Download, the data is in the DownloadData field. The best way to view it and start the process is use a Text To Columns tool after the Download tool with a "\n" (new line) delimiter and set to "Split to Rows". 

I got 11,027 rows from that process, and for example on record 751 (I put a Record ID tool after the Text to Columns tool to help identify the records as I filter them out during a parsing process), I found the first item of data (351'0) that is in the first row of data on the web page for corn.

Again, parsing this out isn't for the faint of heart (the best way is using RegEx to get there) and I have found that sometimes companies change the format of web pages which will potentially end up breaking your parsing process in the future, but it's what you are stuck with if you want an automated way to get the data.

Hope this helps.

Rod

[JP2]

@RodLight

Thanks for the heads up on the parse journey ahead.  Do you have the workflow example that your getting the data back?  I know it's probably something simple that I am missing but for some reason still not getting anything back.  Error says Unsupported Protocol.

Thank you Rod,

JP

[RodLight]

@JP2

Don't see a way to upload an attachment workflow here. If you want to private message me with your email address, I'll send it over that way.

Rod

[russellclaude]

I should have noted that I got my problem resolved.

I had been sending over a ton of unnecessary header values from the header tab in the download tool. I simply needed to uncheck all of the boxes in the "And values from these fields" box which were checked by default. Once I stopped sending stoopid headers to Facebook, they started treating my request with much more decorum!

I am also making heavy use of REGEX and some knowledge of HTML/DOM.

I hope this helps someone. -rh

[spencermckeehan]

Hello,

I need to use the Download Tool for a web scraping task.  The web page I want to access delivers a simple pop up asking for a user id and password to access the URL.

I've tried entering the necessary credentials in the credentials area of the Download tool but get this output:

!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>401 Authorization Required</title> </head><body> <h1>Authorization Required</h1> <p>This server could not verify that you are authorized to access the document requested. Either you supplied the wrong credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required.</p> </body></html>

The URL I am using was originally intended for an Excel Web Query.  When I paste the URL into a browser I'm presented with a simple web page and the table of data I want to download.

Any idea how to properly pass my credentials?

[Mike_D2]

The Icon for Download has changed plus it no longer is part of the Connector group but has moved to the Developer group. 

[mintoi]

I am trying to use the PUT functionality of the Download tool but I get the following error:

HTTP/1.1 415 Unsupported Media Type
date: Thu, 13 May 2021 15:18:28 GMT
content-type: application/json; charset=utf-8
server: Kestrel
transfer-encoding: chunked
content-encoding: gzip
vary: Accept-Encoding
request-context: appId=cid-v1:71a79281-e...

Any thoughts on what I may be doing wrong?

[lepome]

@mintoi 

Does the API you're contacting accept gzip files?