Alteryx designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.

Download tool help!

Highlighted
8 - Asteroid

Dear All,

 

I need some help to configure the Download tool. 

 

The website I need to download from is similar to this one. It has a search function and the data won't display without searching. Is it possible to download the data? 

 

https://apps.ams.usda.gov/pdp

 

Thanks a lot!

Highlighted
Alteryx
Alteryx

Hi @ipeng,

 

it looks like the web interface is hiding the actual query to retrieve the data, so I don't think you can use the Download tool in this case.

Easiest thing to do would be to export the whole dataset in Excel and work from there in Alteryx.

Or, the website you mentioned also exposes APIs (documentation link here https://mymarketnews.ams.usda.gov/mars-api ) where you'll find all the info you need to configure the Alteryx Download tool properly.

 

Hope this helps.

 

Giuseppe

Highlighted
Alteryx Certified Partner
Alteryx Certified Partner

T

here are a whole host of tools out there that allow you to monitor the web-calls made when you hit 'submit' or refresh a page. The easiest way to do this is to right click and hit 'Inspect', a console will open on the right side, navigate to the 'network' tab and then hit 'XHR'. When you next load the page which contains the data you want, the console will populate with the web calls that are being made.

 

From this, it may be possible to decipher the calls made and convert this into an Alteryx workflow. In this instance that seems like it could be possible; the images below provide some detail on why I believe that to be the case...

 

So you can see a POST request is being made...

 

2019-01-14_16-05-01.png

 

The reason a POST request is being made rather than a traditional GET is it seems that what you check on the 'search' menu is being used to query the data, this stuff would represent your 'body'. If you choose 'View Source', this is the structure you should use for your body.

 

2019-01-14_16-06-56.png

 

Another key piece when performing web-scraping are your headers, now not all of these will be necessary. Traditionally, if you are performing a POST requesst, then the 'Content-Type' is the most important header, as that informs how the body is structured.

 

2019-01-14_16-08-16.png

 

In theory, you should be able to get this to work with this information, but, it may be quite tedious and there are no guarantees you will get it to work, but I just wanted to provide this information that it could be possible.

 

Ben

Labels