Web scrapping with multiple action

[HandsomeJih]

Hi everyone,

I have set up a few workflow recently thanks to the help of you guys.

Right now I am trying to build a web scrapping workflow that can 1) log in the website 2) input value in the search bar withing the website and 3) grab the value of the search result.

I have done a few studied about the download function, but is it possible to make username and password a variable that can be change by inputting different value? or I have to just change the workflow once my password expire?

And is alteryx able to support what I want to achieve up there? Or script have to be written for that?

[BrandonB · Accepted solution]

Hi @HandsomeJih 

The download tool is perfect for API calls and web scraping and it can dynamically pass values into API's. However, when it comes to interacting with a webpage and entering text in specific locations, you may want to take a different approach. There is a great write up on using Alteryx and Selenium together to do exactly what you are looking to do. 

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Python-Code-Tool-Web-Scraping-Dynamic-...

Selenium can be used to do exactly what you are looking for and the script provided in the link above can be used as a template which will point you in the right direction. All that you need to do is add the code for what values you want to pass to what text boxes on the page. 

[HandsomeJih]

Thanks a lot, now I have some direction to work on.