community
cancel
Showing results for 
Search instead for 
Did you mean: 

Alteryx Designer Knowledge Base

Definitive answers from Designer experts.
Upgrade Alteryx Designer in 10 Steps

Debating whether or not to upgrade to the latest version of Alteryx Designer?

LEARN MORE

Tool Mastery | Python

Sr. Data Science Content Engineer
Sr. Data Science Content Engineer
Created on
Python.png

This article is part of the Tool Mastery Series, a compilation of Knowledge Base contributions to introduce diverse working examples for Designer Tools. Here we’ll delve into uses of the Python Tool on our way to mastering 
the Alteryx Designer: 

 

Python is one of the fastest growing programming languages in the world and is used for a wide variety of applications ranging from basic data manipulation to data science and software development. With the release of 2018.3 comes the long-awaited and highly anticipated Python Tool! Much like the R-Tool, the Python Tool allows you to seamlessly run code as a part of your Alteryx workflow. Also like the R-Tool, you will need to have some coding experience with the named language in order to use this tool to its maximum potential. In this Tool Mastery Article, we will introduce you to the fundamentals for using this tool.

 

When you first drop the Python Tool on to your Canvas you will see the following screen in the tool’s configuration window. This is a reminder to run your workflow whenever you connect your Python Tool to a new input data source. This pulls the input data into the Python Tool so that you can bring it into your Python Code.

 

2018-11-19_8-08-34.png

 

 

As described in this text, to get the Jupyter Notebook interface up and running, all you need to do is wait. IT takes a couple seconds for the Jupyter Notebook interface to get served the first time you open a Python Tool in an instance of Designer. The message you first see will be replaced with a Jupyter notebook interface.

 

 2018-11-19_8-10-31.png

 

 

For a general introduction to Jupyter Notebooks, please review their Beginner's Guide documentation. 

 

 

The first coding step in using the Python Tool is to import the Alteryx API package, which allows you to pass data between the Alteryx Engine and Python Tool. If you plan on reading in data from the Alteryx Engine or pushing data out to the Engine from the Python Tool, your code should start with:

 

from ayx import Alteryx

 

This piece of code is so fundamental it is automatically populated in the first cell of the Python Tool!

 

2018-11-19_8-10-31.png

 

 

To run an individual cell in the Python Tool, you click the play button in the top toolbar, or you can use the keyboard shortcut: shift + return.

 

2018-11-19_8-14-33.png

 

 

In addition to the ayx package, the Python Tool comes with a few python packages loaded by default. These packages are listed in the help documentation and primarily relate to Data Science. There is also a great article that reviews the functionality of each of these pre-installed packages. To load a package that is already installed, you can use the import command, as you would when creating a Python Script outside of Alteryx. If you would like to install a python library that is not included with the tool by default, you can use the Package.installPackages() function.

 

 2018-11-19_8-18-33.png

 

 

The little * asterisk where the cell number is usually displayed means that the cell is currently running.

 

On the success of installing a package, you will see some variation of the following messages related to dependencies and the version of the package installed.

 

2018-11-19_8-28-24.png

 

 

Optional Follow Along: If you'd like to follow along with this demonstration, please download the Iris Dataset attached to this article!

 

If you are bringing in data through the Input Anchor in Alteryx, you will need to run the workflow to make the incoming data available to the notebook. After running the workflow, you can use the Alteryx.read() function to bring the data into Python.

The only argument to this function is the specific connection you are reading in. Like in the R Tool, this argument is a string and will need to have quotations around it.

 2018-07-30_16-03-02.png

To read in this data stream as the variable name data, the code would read:

 

data = Alteryx.read("#1")

 

2018-11-19_8-31-47.png

 

 

If you try to read in data before running the entire workflow, you will likely see this FileNotFoundError:

 

2018-08-08_16-19-00.png

 

The solution is to save the workflow and then run the workflow. The next time you run the code in the cell with the play button, the error should be resolved.

 

Everything read into the Python Tool will be read in as a pandas data frame. This enables greater flexibility for processing the data in Python. You can change the data format after reading it in, but you will need to return any outputs back to a pandas data frame.

 

Now that I have brought in my data, I would like to analyze it. First, I will create a new cell by clicking the plus icon next to the save/create checkpoint button, or I could use the keyboard shortcut B to add a cell below my current cell.

 

2018-11-19_8-32-46.png

 

 

Other useful cell and notebook functions can also be found in this toolbar to the right of the insert cell below button. From left to right, the buttons are Save, Add a Cell 2018-08-07_12-13-14.png,  Cut Cell(s) 2018-08-07_12-13-57.png, Copy Cell(s))  2018-08-07_12-15-11.png, Paste Cell(s) 2018-08-07_12-16-07.png, Move Cell(s) Up 2018-08-07_12-16-36.png, Move Cell(s) Down 2018-08-07_12-17-34.png,  Run 2018-08-07_13-24-47.png,  Stop 2018-08-07_13-27-59.png, Restart the Kernel 2018-08-07_14-08-08.png, and Restart the Kernel and Rerun the Notebook 2018-08-07_14-09-41.png. All of these buttons have associated keyboard shortcuts. You can see a full list of Jupyter Notebook keyboard shortcuts by navigating to Help > Keyboard Shortcuts in the top toolbar.

 

 

For this demonstration I want to run cluster analysis on the infamous Iris data set, so in my new cell I will load the KMeans function from the Sci-kit learn Python module (included with the Alteryx Python Tool Installation), and write some simple code to create clusters and print the resulting cluster labels.

 

2018-08-07_11-27-41.png

 

Now, I can visualize my clusters with the matplotlib.pyplot python library (also included with the Python Tool by default).

 

opt2.png

 

Finally, writing an output from the Python Tool can be done using the with Alteryx.write() function. This function is currently only supported for pandas data frames. If you attempt to write something out other than a data frame, you will get the following TypeError.

 

2018-08-07_11-45-24.png

 

This error can be resolved by converting your output to a pandas data frame. If you are not yet familiar with pandas data frames, you might find the introduction to pandas data structures or the 10 minutes to pandas documentation helpful. Once you write the code with Alteryx.write() in the Python Tool, you will need to run the entire workflow to see the results in the output anchors of the tool.

 

2018-08-07_11-56-22.png

 

Now, all that is left to do is run the workflow, and the results will be populated in anchor 1 of the Python Tool Outputs.

 

With this overview, I hope you feel comfortable reading in, writing out, and processing data in the Python Tool. The only limits now are your imagination!

 

 

Things to know and Future Updates!

 

  • Starting with 2018.4, you can load externally created python scripts and Jupyter notebooks.
  • Metadata will not consistently populate in downstream tools for data coming out of the Python Tool.
  • There is an implicit type conversion from Boolean to integer on reading data into the Python Tool. Likewise, there is another implicit type conversion from Boolean to integer on writing out from the tool.
  • Starting with 2018.4, you now have the ability to set column data types when writing an output.
  • Only Pandas Data frames are currently supported for reading and writing out. You can not currently write out a plot, or read in and write out spatial objects.
  • Question Constants are not currently supported.

 

If you have any feedback for us on this tool, please post to the Product Ideas Page! Our Product Managers are very active here and would love to see any ideas for features or limitations within the Tool you encounter.

 

By now, you should have expert-level proficiency with the Python Tool! If you can think of a use case we left out, feel free to use the comments section below! Consider yourself a Tool Master already? Let us know at community@alteryx.com if you’d like your creative tool uses to be featured in the Tool Mastery Series.

 

Stay tuned with our latest posts every #ToolTuesday by following @alteryx on Twitter! If you want to master all the Designer tools, consider subscribing for email notifications.

Attachments
Comments
Alteryx Certified Partner
Alteryx Certified Partner

Hey @SydneyF, this is a fantastic feature and thank you for the writeup. Quick question, if I install a package via the Python tool in Alteryx, does the package install to a separate conda environment or does it play off of my system's default conda environment and version?

Sr. Data Science Content Engineer
Sr. Data Science Content Engineer

Hi @michael_treadwell,

 

Thank you for the question! The Python Tool does create a separate Python environment in your Alteryx directory under \bin\Miniconda3 called PythonTool_venv. All packages installed via the Python Tool in Alteryx will be installed in the PythonTool_venv environment, separate from your system's default Conda environment.  

Meteoroid

Can one write custom API calls using the python tool? I have a custom API call written in python so want to know whether or not I can just have the same code in the python tool and it would work? 

Sr. Community Content Manager
Sr. Community Content Manager

@shouvikdas Yes - I have successfully worked with the Quip Automation API with the Python tool.

 

Clicking off and then on doesn't work for me.

 

000010.jpg

Sr. Data Science Content Engineer
Sr. Data Science Content Engineer

Hi @JakeDeJong,

 

Can you please try clicking on and off the Python Tool a couple more times? If the behavior persists, please open a ticket with Alteryx Support by emailing support@alteryx.com.

 

I did x2.   Thank you


Thanks so much for this tool; something we have been waiting for. One question though:

 

"You can not load a saved Jupyter Notebook into the Tool."

 

Does this mean "not yet", i.e., are there plans for enabling this feature? This would really help to "plug in" python code developed from third parties or other teams.

Alteryx Alumni (Retired)

@oliver_huber

 

Hi Oliver,

 

Loading external Notebook files is something that we are actively working towards supporting.

 

Stay tuned!

Meteor

How would I go about performing a database connection via the python tool to execute a sql statement? I would normally use the command tool to execute the code below... can it be done via the Python tool now?

 

import psycopg2

conn_string = "dbname='mydb' port='myport user='myuser' password='mypwd' host='myhost'"
con = psycopg2.connect(conn_string);
sql= """
Insert into targettable (
select * from sourcetable where mycriteria 
IN(select mycriteria from myothertable where country='DE')
); """

sql2="""
Delete from sourcetable 
where mycriteria IN(
select criteria from myothertable where country='DE'); COMMIT;"""
cur = con.cursor()
cur.execute(sql)
cur.execute(sql2)
con.close()

Sr. Community Content Manager
Sr. Community Content Manager

@ecastruita substitute Alteryx.installPackages("psycopg2") in place of your first import statement and you should be good to go.

Meteor

Kinda cool... I was able to run two different sql statements, an insert and a delete statements successfully... so that's cool. But the workflow had a super weird error that I'm certain is a bug:

Error: Python (1): [NbConvertApp] Converting notebook C:\Users\user_id\AppData\Local\Temp\0c30e165-9420-4c12-a252-2155e7c44786\1\workbook.ipynb to html
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] ERROR | Timeout waiting for execute reply (30s).
Traceback (most recent call last):
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 324, in _wait_for_reply
    msg = self.kc.shell_channel.get_msg(timeout=timeout)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\jupyter_client\blocking\channels.py", line 57, in get_msg
    raise Empty
queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Alteryx\bin\Miniconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\Alteryx\bin\Miniconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Program Files\Alteryx\bin\Miniconda3\PythonTool_Venv\Scripts\jupyter-nbconvert.EXE\__main__.py", line 9, in <module>
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\jupyter_core\application.py", line 266, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\traitlets\config\application.py", line 658, in launch_instance
    app.start()
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 325, in start
    self.convert_notebooks()
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 493, in convert_notebooks
    self.convert_single_notebook(notebook_filename)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 464, in convert_single_notebook
    output, resources = self.export_single_notebook(notebook_filename, resources, input_buffer=input_buffer)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 393, in export_single_notebook
    output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 174, in from_filename
    return self.from_file(f, resources=resources, **kw)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 192, in from_file
    return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\html.py", line 85, in from_notebook_node
    return super(HTMLExporter, self).from_notebook_node(nb, resources, **kw)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\templateexporter.py", line 280, in from_notebook_node
    nb_copy, resources = super(TemplateExporter, self).from_notebook_node(nb, resources, **kw)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 134, in from_notebook_node
    nb_copy, resources = self._preprocess(nb_copy, resources)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 311, in _preprocess
    nbc, resc = preprocessor(nbc, resc)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\base.py", line 47, in __call__
    return self.preprocess(nb, resources)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 262, in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\base.py", line 69, in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 280, in preprocess_cell
    reply, outputs = self.run_cell(cell, cell_index)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 348, in run_cell
    exec_reply = self._wait_for_reply(msg_id, cell)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 337, in _wait_for_reply
    raise exception("Cell execution timed out")
TimeoutError: Cell execution timed out

Comet

Thank you Sydney and team, this new tool will be very helpful for my team!

Atom

I am having trouble in instantiating the notebook in designer. What does click off the tool mean? I am not sure where exactly the click needs to occur. Any help is highly appreciated.

Moderator
Moderator

@KUDORJE - try clicking on any blank space on the workflow canvas, then back onto the Python tool to configure it

 

v5JLbz

Meteor

I'm very excited about the Python tool, and this writeup is great! Thank you!

Meteor

And my issue of the execute.py component timing out at 30 secs has been addressed and a temporary fixed identified. 

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Python-Tool-Timeouts-When-Running-Work...

Meteoroid

After successfully installing psycopg2 using Alteryx.installPackages(), it reports that my pip should be upgrade.

 

You are using pip version 9.0.1, however version 18.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

 

I know my pip is version 18.1 in my system's environment but that is separate from Alteryx's env. How does one resolve this?

 

This a great post by the way. Thanks!

Sr. Data Science Content Engineer
Sr. Data Science Content Engineer

Hi @dantvli,

 

Thank you for your question, can you please post it to the Designer Forum? This will give it higher visibility on the Community, and allow you to engage with some of our Python experts. To get you started, the Python Tool creates its own virtual environment to ensure that any package installations or other modifications do not impact any Python Installations outside of Alteryx and vice versa. At this point, I would not suggest trying to update the pip version used in the Python Tool, as it may impact how Alteryx.installPackages() works.

 

Thanks!

Meteoroid

Thank You @SY. This tool is a great addition to Alteryx.

However, I am struggling to access workflow constants inside Python Tool. I have a requirement to use Workflow and Temporary directory inside python code. Is there any direct way to do this?

 

I recognize that we can put the formula tool as input in python and adjust the constants there but just wondering if there is any direct way.

Sr. Data Science Content Engineer
Sr. Data Science Content Engineer

Hi @nishanttayal,

 

Have you upgraded to 2018.4? Workflow constants were added to the Python tool in this release, and you should be able to access them with the function Alteryx.getWorkflowConstant(). After upgrading, try running Alteryx.help() for additional documentation.

 

Thanks!

 

Sydney

Bolide

Hey team,

 

I went through tutorial above and finally managed to get plot. 

You're missing 
plt.show()

command to show scatterplot and closing ']' to populate clusData array.

 

I always appreciate code snippets as text in tutorial since python case sensitive... in the mean time here it is:

 

[1]:
from ayx import Package, Alteryx
Package.installPackages(['nltk'])

[2:]
data = Alteryx.read("#1")
data

[3:]
from sklearn.cluster import KMeans
clusData = data[['SepalLengthCm','SepalWidthCm','PetalLengthCm','PetalWidthCm']]
kmeans = KMeans(n_clusters=3,random_state=0).fit(clusData)
print(kmeans.labels_)

[4:]
import matplotlib.pyplot as plt
plt.scatter(clusData['SepalLengthCm'],clusData['SepalWidthCm'],c=kmeans.labels_)
plt.title=("Iris Clusters")
plt.show()

[5:]
import pandas as pd
labelsDF = pd.DataFrame(data=kmeans.labels_) 
Alteryx.write(labelsDF, 1)

Few things came to mind using the python tool in 2018.4.5:

- data is is not persistent? So when editing code, I noticed clusData suddenly was not defined anymore unless I ran whole workflow again.

- It would be good to clarify the buttons - 'Run' just runs cell while 'FWD' button runs all code for all cells...

- I had to put cursor in cell4 and hit 'Run' to finally get the plot to show up? Otherwise I just see

<Figure size 640x480 with 1 Axes>

- alteryx.write(plt,2) as image or html?

- I am missing (# button) button to quickly (un)comment code but that may be Jupyter limitation

- Optional hide [out:] I don't care seeing long details of installing package messages (maybe once)

 

All in all, great release of the py tool and look forward delving into it more!

Atom

Yesterday I created a simple pipeline that took the Master Store File - CO from the sample data and ran it into the Python tool. I just did simple normalization of data using Pandas and then outputted it to make a chart of that data. 

 

Today I created another pipeline into the same Python tool with input #2 and followed this example successfully, however upon running the workflow, I saw that there was an error for the input #1. How does the jupyter notebook interact with Alteryx? The error I am seeing is that there is no valid metadata for pipeline #1, and it is requiring me to re-run the jupternotebook code I ran yesterday. Do I have to rererun the code within the Python tool everytime I disconnect from Alteryx? 

 

Everything worked fine yesterday...and I have an Alteryx.write(dfs,1) statement as shown in the screenshot, but it keeps saying that there is no valid metadata for the outgoing connection 1.Capture1.PNG

Atom

To clarify, if I am in the Python tool and run all of the code, then run the workflow and don't click anywhere else, everything works. But if I click inside the python tool just to look at something and not make any changes, then click outside of the tool into canvas white space, an error is thrown saying there is no valid metadata for output number one. 

Meteoroid

Hi @honeypot ,

Python tool works like any other tool in Alteryx and if you are re-opening the workflow then you need to execute the workflow to get the valid output from it.

And when you click on Python Tool to see the code it initiates the jupyter notebook for editing and reset the previous code therefore it doesn't find the metadata and we need to re-execute it.

Hope it helps. 

Atom

@nishanttayal Yes this helps! Sorry yesterday was my first day using the tool (and free trial)!! 

Alteryx Certified Partner

When will Plots/Charts be supported to write out? I am building ML modules and would like to export the feature impact and correlation matrix plots? As mentioned I tried writing those out and got an error.