Web scrapping with GET method

Question

Hi everyone,

I am new user of Alteryx, and I would like to scrap a HTML page with Alteryx.

I also began to learn about webscraping.

So, the website that I would like to scrap use a GET method request. I don't know how to use or to set this request.

I've used the download tools but I did not manage to collect the code of the page with informations.

Thank you for helping me to scrap a website.

hellyars · Answer

@ByranCarter12

My experience with web scrapes...  You can do it in Alteryx, but it can be a bit cumbersome (and I had a few ACE's from one of Alteryx's partners help setup my Alteryx derived solution).  UI Path can be much quicker.  Plus, it exports to Excel.  You can then use Alteryx for further analysis.  But, without knowing your target URL it is a bit difficult to troubleshoot.   That said, I'd trust @afv2688 first.

ByranCarter12 · Answer

@afv2688, Thank you for your answer. I have read the post and I tried to use the download Tool but It's seems to not working.

In fact, I need to specify on the website search criteria like location, brands... to give me information.

Usually when you do a search on a website, you get a URL like www.website_name.com/?country=UK&brands=whatever

But in my case, I don't get these informations in the URL. So I don't know if there is a script that executes these criteria in separate ways.

In any case, I don't get the result of the search in the HTML code but rather the search criteria for example: input type = "checkbox"
Here I do not know if I should create a specific request to integrate it when downloading the code. That's why I was asking if it possible to parameterize a request with the GET method.

Thank you.

hellyars · Answer

It also helps to set the headers section in the Download Tool. You can use Chrome's Inspect (Network>>All>>Headers) feature to get a given page's information.

Here is good article on it (from an Alteryx use case perspective).  It focuses on the POST and GET methods.