Using Python code to download files then using Alteryx to use those files

[Nmassarone]

Hello I'm a little newer to python inside Alteryx I do have some experience with python outside of Alteryx and am somewhat good with Alteryx I have a question about using Python and Alteryx together.

I created python code (using PyCharm) that goes to a webpage and downloads an excel file to a folder. It uses selenium, os, and time functions in the code.

Really what I'm looking to do is to have that Python code to run before the Alteryx workflow to download the file and then the workflow will look in that folder directory find that file and use it in the workflow I have.

The Python code doesn't really need to output anything into the workflow im just looking to make it simple....

Ideally what im looking to do is we have other people on our team that run the workflows right now we have to go to a couple different websites and download files then put them in a folder and then we run the Alteryx workflow that finds the files in that directory. What i did is created a Python script to go to all the webpages get the files and put them in that folder. Then i want the Alteryx workflow to run to get those files and use them in the workflow. that way someone else on my team can just hit run and it will run the python code to get the files then the Alteryx workflow to run the workflow.

im just wondering if this is possible to have the python script inside the python tool run FIRST before the rest of the workflow. 

Also is it even possible to use selenium in the python tool in the workflow and just have the python tool in its own container not connected to anything?

Also another question for the packages that have to be installed do i have to have the Package.installPackages(['selenium']) in there everytime the workflow is ran or once its installed is it installed for good an i can just use the from selenium import webdriver in the python tool?

just looking for a little help on this if something like this is possible?

thank you

[FinnCharlton · Accepted solution]

Hi @Nmassarone :

• To make the Python tool run first, you have a couple of options. If you are using the latest Alteryx release, you can use control containers to achieve this : https://help.alteryx.com/20231/designer/control-container-tool. If not, I would suggest using chained apps to achieve this: https://community.alteryx.com/t5/Interactive-Lessons/Chaining-Analytic-Apps/ta-p/243120 .
• You can definitely use Selenium in Alteryx, and you yes you can just have the python tool on it's own without connections.
• You should only need to install Selenium once on each separate machine you run the workflow on. I don't think leaving it in the script will cause any harm.

[Nmassarone]

ok perfect thank you @FinnCharlton ... i am currently not on the latest version unfortunately our company is a little back I'm on 2021.3.5

Thank you for all the information ill look into using the chained apps. 

Do you happen to know since i am on a company computer they kind of limit what we can do ( I do not have admin privileges) do you happen to know if I would be alright to do the package installs in Alteryx or does all that need admin rights?

thank you again for your help and reply appreciate it.

[BS_THE_ANALYST]

@Nmassarone when downloading the packages, I think you'll need to do something like !pip install blah blah --user (https://stackoverflow.com/questions/42988977/what-is-the-purpose-of-pip-install-user). 

If you're downloading files using the Python tool, I'd also advise writing a few more lines in the Python script to output the file paths for the file(s) you're wanting to bring into Alteryx. Create a dataframe from the file paths then use Alteryx.write(df,1) to output the filepaths (simulating a directory tool in alteryx). 

This would form the initial part of your workflow sequencing. From the output anchor of the Python tool, you can use dynamic inputs or batch macros to bring those files in. I don't see the need for control containers, chained apps (although they are super useful!)

I'll attach a workflow where I've accomplished some of the things you'll need. 
(note: don't point the directory tool to a folder that contains .xlsx files, it will create CSVs copies of them lol - it's just so you can copy parts of the python tool)

All the best,

BS

[Nmassarone]

Thank you @BS_THE_ANALYST ... do you happen to know if you need to be running Alteryx as admin to use the !pip install ? our company kind of blocks running as admin we cannot open Alteryx as admin we would need someone from our IT department to do it so wasnt sure if there was a work around for installing the packages.

Thank you for the help on this ill look into the StackOverflow

Appreciate the help!

[BS_THE_ANALYST · Accepted solution]

@Nmassarone My advice it to try to download a package.

Check this article (last post):  https://community.alteryx.com/t5/Alteryx-Designer-Desktop-Discussions/Install-python-packages-in-Alt...
This implies it is possible.

All the best,

BS
 

[Nmassarone]

perfect the !pip install --user (package) worked for me was able to download the package and run the workflow with the python code

appreciate your help on this thank you!

[BS_THE_ANALYST]

@Nmassarone Great stuff. Let me know if you have success with outputting the directory that you've downloaded your files to using your Python script. I think the bottom section of my code is doing that part with the OS library. 

All the best,

BS