Switch python virtual env

[gillsuki]

Hi

My understanding Alteryx Designer uses miniconda underneath the hood to manage the python environment.

You can use the Package.installPackages() to load user specific libraries into the base environment.

Is it possible to switch to an alternative virtual environment into which I can install my user specific libraries?

Regards

Suki

[Gaurav_Dhama_]

See if this gives you a headstart.

Virtual Environments - Alteryx Community

[gillsuki]

Thx.

So I create my custom virtual environment using conda on the same machine running Alteryx designer.

How do I then switch to this new virtual environment when running the Jupyter script in Designer?

Also in the context of Alteryx Server, I presume I would need to manually setup new virtual env using conda on the server before scheduling the workflow to run.

[BS_THE_ANALYST]

@gillsuki please use the Jupyter Flow tool for this https://help.alteryx.com/current/en/designer/tools/laboratory/jupyter-flow.html it'll allow you to call on the .py file you want to run and select the appropriate virtual environment. 

Here's a nice article on it: https://community.alteryx.com/t5/Data-Science/Introducing-the-Jupyter-Flow-Tool/ba-p/779654 

[BS_THE_ANALYST]

@Gaurav_Dhama_ regarding your second question about Server, your thoughts seem sensible. You could install the appropriate libraries/environment on there (not sure if you have a single-node for your server? Could be some further thinking required there). That being said, a python virtual environment is just a folder and Alteryx ships with a python.exe (which will already be on the server machine). Theoretically, you should be able to leverage that .exe & just package the virtual environment when uploading to server. Some testing will be needed here 😀

With that being said, there's a section https://help.alteryx.com/current/en/designer/tools/laboratory/jupyter-flow.html at the bottom of this article for server

I checked the other article I sent you: https://community.alteryx.com/t5/Data-Science/Introducing-the-Jupyter-Flow-Tool/ba-p/779654 and there's lots of promising references to server. I'd follow that as a guide

  
I'm curious to see how this evolves.

[apathetichell]

In my experience ---  and @BS_THE_ANALYST  ---  I'll defer to you if your experience is different --- because Alteryx runs Alteryx python connectors in the python environment --- Alteryx manages the python venv. You cannot port/execute scripts in a user managed Python Venv in Alteryx. My recommendation is to execute python code via the run command tool. My recommendation is to manage all venvs in shell via uv --- and to script the necessary uv sync/uv run commands so that you can readily configure cross platform venv management (ie desktop/server).

[BS_THE_ANALYST]

@apathetichell the Jupyter Flow tool worked for me on a previous project. Needed to call on different virtual environment and run a notebook. I followed this guide https://community.alteryx.com/t5/Data-Science/Introducing-the-Jupyter-Flow-Tool/ba-p/779654 

I think the Run Command tool route is really nice route @apathetichell. I think there's a great blog opportunity here to showcase this (or a knowledge article). I haven't tried this route yet, but it makes a lot of sense.

[apathetichell]

@BS_THE_ANALYST --- awesome --- if I get an Alteryx license again --- I'll play around with it. My hunch is that uv is going to be levels faster for anything with more package requirements --- but it's good to see this is an option.

[BS_THE_ANALYST]

@apathetichell I see you're still active with the community despite the lack of a license. Is it work reaching out to SparkED https://www.alteryx.com/sparked/learning-programs/independent-learners#AdaptForm ? Not sure if you've tried this yet.

[BS_THE_ANALYST]

@gillsuki I've had a look into this now. I think this answers your post & your other post around capturing the output in the terminal. 

I've created a virtual environment here:

There's a .bat file in there which contains the commands to activate the virtual environment and then run the python script. There's also an output log file which gets generated as part of the .bat file. The content of the .bat file look like this:

@echo off
REM Change to the directory of the virtual environment
cd /d "C:\Users\benja\Downloads\CommunityVenv"

REM Activate the virtual environment
call Scripts\activate

REM Install numpy
pip install numpy

REM Run the Python script and capture output
python pythonfile.py > output.log 2>&1

REM Deactivate the virtual environment
deactivate

The python script is simple (it's just listing the packages in the virtual environment)

import pip
import subprocess
import sys
print("Hello World")

def pip_list():
    args = [sys.executable, "-m", "pip", "list"]
    p = subprocess.run(args, check=True, capture_output=True)
    return p.stdout.decode()

print(pip_list())

The workflow:

 This just calls on the .bat file and then reads the Output log which the .bat file generates into the results window.