Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Tool Mastery

Explore a diverse compilation of articles that take an in-depth look at Designer tools.
Become a Tool Master

Learn how you can share your expertise with the Community

LEARN MORE

Tool Mastery | Python

SydneyF
Alteryx Alumni (Retired)
Created
Python.png

This article is part of the Tool Mastery Series, a compilation of Knowledge Base contributions to introduce diverse working examples for Designer Tools. Here we’ll delve into uses of the Python Tool on our way to mastering
the Alteryx Designer:

 

Python is one of the fastest growing programming languages in the world and is used for a wide variety of applications ranging from basic data manipulation to data science and software development. With the release of 2018.3 comes the long-awaited and highly anticipated Python Tool! Much like the R-Tool, the Python Tool allows you to seamlessly run code as a part of your Alteryx workflow. Also like the R-Tool, you will need to have some coding experience with the named language in order to use this tool to its maximum potential. In this Tool Mastery Article, we will introduce you to the fundamentals for using this tool.

 

When you first drop the Python Tool on to your Canvas you will see the following screen in the tool’s configuration window. This is a reminder to run your workflow whenever you connect your Python Tool to a new input data source. This pulls the input data into the Python Tool so that you can bring it into your Python Code.

 

2018-11-19_8-08-34.png

 

 

As described in this text, to get theJupyter Notebook interface up and running, all you need to do is wait. IT takes a couple seconds for the Jupyter Notebook interface to get served the first time you open a Python Tool in an instance of Designer. The message you first see will be replaced with a Jupyter notebook interface.

 

2018-11-19_8-10-31.png

 

 

For a general introduction to Jupyter Notebooks, please review their Beginner's Guidedocumentation.

 

 

The first coding step in using the Python Tool is to import the Alteryx API package, which allows you to pass data between the Alteryx Engine and Python Tool. If you plan on reading in data from the Alteryx Engine or pushing data out to the Engine from the Python Tool, your code should start with:

 

from ayx import Alteryx

 

This piece of code is so fundamental it is automatically populated in the first cell of the Python Tool!

 

2018-11-19_8-10-31.png

 

 

To run an individual cell in the Python Tool, you click the play button in the top toolbar, or you can use the keyboard shortcut: shift + return.

 

2018-11-19_8-14-33.png

 

 

In addition to the ayx package, the Python Tool comes with a few python packages loaded by default. These packages are listed in the help documentation and primarily relate to Data Science. There is also a great article that reviews the functionality of each of these pre-installed packages. To load a package that is already installed, you can use the import command, as you would when creating a Python Script outside of Alteryx. If you would like to install a python library that is not included with the tool by default, you can use the Package.installPackages() function.

 

2018-11-19_8-18-33.png

 

 

The little * asterisk where the cell number isusually displayed means that the cell is currently running.

 

On the success of installing a package, you will see some variation of the following messages related to dependencies and the version of the package installed.

 

2018-11-19_8-28-24.png

 

 

Optional Follow Along:If you'd like to follow along with this demonstration, please download the Iris Dataset attached to this article!

 

If you are bringing in data through the Input Anchor in Alteryx, you will need to run the workflow to make the incoming data available to the notebook. After running the workflow, you can use the Alteryx.read() function to bring the data into Python.

The only argument to this function is the specific connection you are reading in. Like in the R Tool, this argument is a string and will need to have quotations around it.

2018-07-30_16-03-02.png

To read in this data stream as the variable name data, the code would read:

 

data = Alteryx.read("#1")

 

2018-11-19_8-31-47.png

 

 

If you try to read in data before running the entire workflow, you will likely see this FileNotFoundError:

 

2018-08-08_16-19-00.png

 

The solution is to save the workflow and then run the workflow. The next time you run the code in the cell with the play button, the error should be resolved.

 

Everything read into the Python Tool will be read in as a pandas data frame. This enables greater flexibility for processing the data in Python. You can change the data format after reading it in, but you will need to return any outputs back to a pandas data frame.

 

Now that I have brought in my data, I would like to analyze it. First, I will create a new cell by clicking the plus icon next to the save/create checkpoint button, or I could use the keyboard shortcut B to add a cell below my current cell.

 

2018-11-19_8-32-46.png

 

 

Other useful cell and notebook functions can also be found in this toolbar to the right of the insert cell below button. From left to right, the buttons are Save, Add a Cell2018-08-07_12-13-14.png, Cut Cell(s) 2018-08-07_12-13-57.png, Copy Cell(s))2018-08-07_12-15-11.png,Paste Cell(s)2018-08-07_12-16-07.png, Move Cell(s) Up2018-08-07_12-16-36.png, Move Cell(s) Down2018-08-07_12-17-34.png, Run2018-08-07_13-24-47.png, Stop2018-08-07_13-27-59.png, Restart the Kernel2018-08-07_14-08-08.png,and Restart the Kernel and Rerun the Notebook2018-08-07_14-09-41.png. All of these buttons have associated keyboard shortcuts. You can see a full list of Jupyter Notebook keyboard shortcuts by navigating to Help > Keyboard Shortcuts in the top toolbar.

 

 

For this demonstration I want to run cluster analysis on the infamous Iris data set, so in my new cell I will load the KMeansfunction from the Sci-kit learn Pythonmodule(included with the Alteryx Python Tool Installation), and write some simple code to create clusters and print the resulting cluster labels.

 

2018-08-07_11-27-41.png

 

Now, Ican visualize my clusters with the matplotlib.pyplot python library (also included with the Python Tool by default).

 

opt2.png

 

Finally, writing an output from the Python Tool can be done using the with Alteryx.write() function. This function is currently only supported for pandas data frames. If you attempt to write something out other than a data frame, you will get the following TypeError.

 

2018-08-07_11-45-24.png

 

This error can be resolved by converting your output to a pandas data frame. If you are notyet familiar with pandas data frames, you might find the introduction to pandas data structuresor the 10 minutes to pandas documentationhelpful.Once you write the code with Alteryx.write() in the Python Tool, you will need to run the entire workflow to see the results in the output anchors of the tool.

 

2018-08-07_11-56-22.png

 

Now, all that is left to do is run the workflow, and the results will be populated in anchor 1 of the Python Tool Outputs.

 

With this overview, I hope you feel comfortable reading in, writing out, and processing data in the Python Tool. The only limits now are your imagination!

 

 

Things to know and Future Updates!

 

  • Starting with 2018.4, you can load externally created python scripts and Jupyter notebooks.
  • Metadata will not consistently populate in downstream tools for data coming out of the Python Tool.
  • There is an implicit type conversion from Boolean to integer on reading data into the Python Tool. Likewise, there is another implicit type conversion from Boolean to integer on writing out from the tool.
  • Starting with 2018.4, you now have the ability to set column data types when writing an output.
  • Only Pandas Data frames are currently supported for reading and writing out. You can not currently write out a plot, or read in and write out spatial objects.
  • Question Constants are not currently supported.

 

If you have any feedback for us on this tool, please post to the Product Ideas Page! Our Product Managers are very active here and would love to see any ideas for features or limitations within the Tool you encounter.

By now, you should have expert-level proficiency with the Python Tool! If you can think of a use case we left out, feel free to use the comments section below! Consider yourself a Tool Master already? Let us know atcommunity@alteryx.comif you’d like your creative tool uses to be featured in the Tool Mastery Series.

 

Stay tuned with our latest posts every#ToolTuesdayby following@alteryxon Twitter! If you want to master all the Designer tools, considersubscribingfor email notifications.

Additional Information

Click on the corresponding language link below to access this article in another language -

Portuguese
Spanish
French
German
Japanese

Attachments
Comments
michael_treadwell
ACE Emeritus
ACE Emeritus

Hey @SydneyF, this is a fantastic feature and thank you for the writeup. Quick question, if I install a package via the Python tool in Alteryx, does the package install to a separate conda environment or does it play off of my system's default conda environment and version?

SydneyF
Alteryx Alumni (Retired)

Hi @michael_treadwell,

 

Thank you for the question! The Python Tool does create a separate Python environment in your Alteryx directory under \bin\Miniconda3 called PythonTool_venv. All packages installed via the Python Tool in Alteryx will be installed in the PythonTool_venv environment, separate from your system's default Conda environment.  

shouvikdas
6 - Meteoroid

Can one write custom API calls using the python tool? I have a custom API call written in python so want to know whether or not I can just have the same code in the python tool and it would work? 

NeilR
Alteryx Alumni (Retired)

@shouvikdas Yes - I have successfully worked with the Quip Automation API with the Python tool.

JDeJong
8 - Asteroid

 

Clicking off and then on doesn't work for me.

 

000010.jpg

SydneyF
Alteryx Alumni (Retired)

Hi @JDeJong,

 

Can you please try clicking on and off the Python Tool a couple more times? If the behavior persists, please open a ticket with Alteryx Support by emailing support@alteryx.com.

 

JDeJong
8 - Asteroid

I did x2.   Thank you


oliver_huber
7 - Meteor

Thanks so much for this tool; something we have been waiting for. One question though:

 

"You can not load a saved Jupyter Notebook into the Tool."

 

Does this mean "not yet", i.e., are there plans for enabling this feature? This would really help to "plug in" python code developed from third parties or other teams.

ColinR
Alteryx Alumni (Retired)

@oliver_huber

 

Hi Oliver,

 

Loading external Notebook files is something that we are actively working towards supporting.

 

Stay tuned!

ecastruita
7 - Meteor

How would I go about performing a database connection via the python tool to execute a sql statement? I would normally use the command tool to execute the code below... can it be done via the Python tool now?

 

import psycopg2

conn_string = "dbname='mydb' port='myport user='myuser' password='mypwd' host='myhost'"
con = psycopg2.connect(conn_string);
sql= """
Insert into targettable (
select * from sourcetable where mycriteria 
IN(select mycriteria from myothertable where country='DE')
); """

sql2="""
Delete from sourcetable 
where mycriteria IN(
select criteria from myothertable where country='DE'); COMMIT;"""
cur = con.cursor()
cur.execute(sql)
cur.execute(sql2)
con.close()

NeilR
Alteryx Alumni (Retired)

@ecastruita substitute Alteryx.installPackages("psycopg2") in place of your first import statement and you should be good to go.

ecastruita
7 - Meteor

Kinda cool... I was able to run two different sql statements, an insert and a delete statements successfully... so that's cool. But the workflow had a super weird error that I'm certain is a bug:

Error: Python (1): [NbConvertApp] Converting notebook C:\Users\user_id\AppData\Local\Temp\0c30e165-9420-4c12-a252-2155e7c44786\1\workbook.ipynb to html
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] ERROR | Timeout waiting for execute reply (30s).
Traceback (most recent call last):
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 324, in _wait_for_reply
    msg = self.kc.shell_channel.get_msg(timeout=timeout)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\jupyter_client\blocking\channels.py", line 57, in get_msg
    raise Empty
queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Alteryx\bin\Miniconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\Alteryx\bin\Miniconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Program Files\Alteryx\bin\Miniconda3\PythonTool_Venv\Scripts\jupyter-nbconvert.EXE\__main__.py", line 9, in <module>
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\jupyter_core\application.py", line 266, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\traitlets\config\application.py", line 658, in launch_instance
    app.start()
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 325, in start
    self.convert_notebooks()
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 493, in convert_notebooks
    self.convert_single_notebook(notebook_filename)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 464, in convert_single_notebook
    output, resources = self.export_single_notebook(notebook_filename, resources, input_buffer=input_buffer)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 393, in export_single_notebook
    output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 174, in from_filename
    return self.from_file(f, resources=resources, **kw)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 192, in from_file
    return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\html.py", line 85, in from_notebook_node
    return super(HTMLExporter, self).from_notebook_node(nb, resources, **kw)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\templateexporter.py", line 280, in from_notebook_node
    nb_copy, resources = super(TemplateExporter, self).from_notebook_node(nb, resources, **kw)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 134, in from_notebook_node
    nb_copy, resources = self._preprocess(nb_copy, resources)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 311, in _preprocess
    nbc, resc = preprocessor(nbc, resc)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\base.py", line 47, in __call__
    return self.preprocess(nb, resources)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 262, in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\base.py", line 69, in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 280, in preprocess_cell
    reply, outputs = self.run_cell(cell, cell_index)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 348, in run_cell
    exec_reply = self._wait_for_reply(msg_id, cell)
  File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 337, in _wait_for_reply
    raise exception("Cell execution timed out")
TimeoutError: Cell execution timed out

cam_w
11 - Bolide

Thank you Sydney and team, this new tool will be very helpful for my team!

KUDORJE
5 - Atom

I am having trouble in instantiating the notebook in designer. What does click off the tool mean? I am not sure where exactly the click needs to occur. Any help is highly appreciated.

SophiaF
Alteryx
Alteryx

@KUDORJE - try clicking on any blank space on the workflow canvas, then back onto the Python tool to configure it

 

v5JLbz

shaynie
8 - Asteroid

I'm very excited about the Python tool, and this writeup is great! Thank you!

ecastruita
7 - Meteor

And my issue of the execute.py component timing out at 30 secs has been addressed and a temporary fixed identified. 

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Python-Tool-Timeouts-When-Running-Work...

dantvli
6 - Meteoroid

After successfully installing psycopg2 using Alteryx.installPackages(), it reports that my pip should be upgrade.

 

You are using pip version 9.0.1, however version 18.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

 

I know my pip is version 18.1 in my system's environment but that is separate from Alteryx's env. How does one resolve this?

 

This a great post by the way. Thanks!

SydneyF
Alteryx Alumni (Retired)

Hi @dantvli,

 

Thank you for your question, can you please post it to the Designer Forum? This will give it higher visibility on the Community, and allow you to engage with some of our Python experts. To get you started, the Python Tool creates its own virtual environment to ensure that any package installations or other modifications do not impact any Python Installations outside of Alteryx and vice versa. At this point, I would not suggest trying to update the pip version used in the Python Tool, as it may impact how Alteryx.installPackages() works.

 

Thanks!

nishanttayal
6 - Meteoroid

Thank You @SY_dup_278. This tool is a great addition to Alteryx.

However, I am struggling to access workflow constants inside Python Tool. I have a requirement to use Workflow and Temporary directory inside python code. Is there any direct way to do this?

 

I recognize that we can put the formula tool as input in python and adjust the constants there but just wondering if there is any direct way.

SydneyF
Alteryx Alumni (Retired)

Hi @nishanttayal,

 

Have you upgraded to 2018.4? Workflow constants were added to the Python tool in this release, and you should be able to access them with the function Alteryx.getWorkflowConstant(). After upgrading, try running Alteryx.help() for additional documentation.

 

Thanks!

 

Sydney

simon
11 - Bolide

Hey team,

 

I went through tutorial above and finally managed to get plot. 

You're missing 
plt.show()

command to show scatterplot and closing ']' to populate clusData array.

 

I always appreciate code snippets as text in tutorial since python case sensitive... in the mean time here it is:

 

[1]:
from ayx import Package, Alteryx
Package.installPackages(['nltk'])

[2:]
data = Alteryx.read("#1")
data

[3:]
from sklearn.cluster import KMeans
clusData = data[['SepalLengthCm','SepalWidthCm','PetalLengthCm','PetalWidthCm']]
kmeans = KMeans(n_clusters=3,random_state=0).fit(clusData)
print(kmeans.labels_)

[4:]
import matplotlib.pyplot as plt
plt.scatter(clusData['SepalLengthCm'],clusData['SepalWidthCm'],c=kmeans.labels_)
plt.title=("Iris Clusters")
plt.show()

[5:]
import pandas as pd
labelsDF = pd.DataFrame(data=kmeans.labels_) 
Alteryx.write(labelsDF, 1)

Few things came to mind using the python tool in 2018.4.5:

- data is is not persistent? So when editing code, I noticed clusData suddenly was not defined anymore unless I ran whole workflow again.

- It would be good to clarify the buttons - 'Run' just runs cell while 'FWD' button runs all code for all cells...

- I had to put cursor in cell4 and hit 'Run' to finally get the plot to show up? Otherwise I just see

<Figure size 640x480 with 1 Axes>

- alteryx.write(plt,2) as image or html?

- I am missing (# button) button to quickly (un)comment code but that may be Jupyter limitation

- Optional hide [out:] I don't care seeing long details of installing package messages (maybe once)

 

All in all, great release of the py tool and look forward delving into it more!

honeypot
6 - Meteoroid

Yesterday I created a simple pipeline that took the Master Store File - CO from the sample data and ran it into the Python tool. I just did simple normalization of data using Pandas and then outputted it to make a chart of that data. 

 

Today I created another pipeline into the same Python tool with input #2 and followed this example successfully, however upon running the workflow, I saw that there was an error for the input #1. How does the jupyter notebook interact with Alteryx? The error I am seeing is that there is no valid metadata for pipeline #1, and it is requiring me to re-run the jupternotebook code I ran yesterday. Do I have to rererun the code within the Python tool everytime I disconnect from Alteryx? 

 

Everything worked fine yesterday...and I have an Alteryx.write(dfs,1) statement as shown in the screenshot, but it keeps saying that there is no valid metadata for the outgoing connection 1.Capture1.PNG

honeypot
6 - Meteoroid

To clarify, if I am in the Python tool and run all of the code, then run the workflow and don't click anywhere else, everything works. But if I click inside the python tool just to look at something and not make any changes, then click outside of the tool into canvas white space, an error is thrown saying there is no valid metadata for output number one. 

nishanttayal
6 - Meteoroid

Hi @honeypot ,

Python tool works like any other tool in Alteryx and if you are re-opening the workflow then you need to execute the workflow to get the valid output from it.

And when you click on Python Tool to see the code it initiates the jupyter notebook for editing and reset the previous code therefore it doesn't find the metadata and we need to re-execute it.

Hope it helps. 

honeypot
6 - Meteoroid

@nishanttayal Yes this helps! Sorry yesterday was my first day using the tool (and free trial)!! 

Inactive User
Not applicable

When will Plots/Charts be supported to write out? I am building ML modules and would like to export the feature impact and correlation matrix plots? As mentioned I tried writing those out and got an error.

D3100
6 - Meteoroid
I'm having trouble installing packages. I have a Python 3.7 installed on my system as my main interpreter. (just so you're aware). So my issue is trying to install packages using the Package.installpackages or the Alteryx.installPackage() methods. I'm getting the following error: In [2]: Alteryx.installPackage('openpyxl') Collecting openpyxl Using cached https://files.pythonhosted.org/packages/ba/06/b899c8867518df19e242d8cbc82d4ba210f5ffbeebb7704c695e68... Collecting jdcal (from openpyxl) Using cached https://files.pythonhosted.org/packages/f0/da/572cbc0bc582390480bbd7c4e93d14dc46079778ed915b505dc494... Collecting et_xmlfile (from openpyxl) Using cached https://files.pythonhosted.org/packages/22/28/a99c42aea746e18382ad9fb36f64c1c1f04216f41797f2f0fa567d... Installing collected packages: jdcal, et-xmlfile, openpyxl Exception: Traceback (most recent call last): File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\pip\basecommand.py", line 215, in main status = self.run(options, args) File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\pip\commands\install.py", line 342, in run prefix=options.prefix_path, File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\pip\req\req_set.py", line 784, in install **kwargs File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\pip\req\req_install.py", line 851, in install self.move_wheel_files(self.source_dir, root=root, prefix=prefix) File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\pip\req\req_install.py", line 1064, in move_wheel_files isolated=self.isolated, File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\pip\wheel.py", line 345, in move_wheel_files clobber(source, lib_dir, True) File "c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\pip\wheel.py", line 323, in clobber shutil.copyfile(srcfile, destfile) File "C:\Program Files\Alteryx\bin\Miniconda3\lib\shutil.py", line 115, in copyfile with open(dst, 'wb') as fdst: PermissionError: [Errno 13] Permission denied: 'c:\\program files\\alteryx\\bin\\miniconda3\\pythontool_venv\\Lib\\site-packages\\jdcal.py' --------------------------------------------------------------------------- CalledProcessError Traceback (most recent call last) in ----> 1 Alteryx.installPackage('openpyxl') c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\ayx\export.py in installPackage(package, install_type, debug, **kwargs) 151 This function will install a package or list of packages into the virtual environment used by the Python tool. If using an admin installation of Alteryx, you must run Alteryx as administrator in order to use this function and install packages. 152 """ --> 153 __installPackages__(package, install_type=install_type, debug=debug, **kwargs) 154 155 c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\ayx\Package.py in installPackages(package, install_type, debug) 112 print(pip_install_result['msg']) 113 if not pip_install_result['success']: --> 114 raise pip_install_result['err'] c:\program files\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\ayx\Utils.py in runSubprocess(args_list, debug) 39 result = subprocess.check_output( 40 args_list, ---> 41 stderr = subprocess.STDOUT 42 ) 43 if debug: C:\Program Files\Alteryx\bin\Miniconda3\lib\subprocess.py in check_output(timeout, *popenargs, **kwargs) 334 335 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, --> 336 **kwargs).stdout 337 338 C:\Program Files\Alteryx\bin\Miniconda3\lib\subprocess.py in run(input, timeout, check, *popenargs, **kwargs) 416 if check and retcode: 417 raise CalledProcessError(retcode, process.args, --> 418 output=stdout, stderr=stderr) 419 return CompletedProcess(process.args, retcode, stdout, stderr) 420 CalledProcessError: Command '['c:\\program files\\alteryx\\bin\\miniconda3\\pythontool_venv\\scripts\\python.exe', '-m', 'pip', 'install', 'openpyxl']' returned non-zero exit status 2. In [ ]:
nishanttayal
6 - Meteoroid

@D3100 Check whether you are running Alteryx as an administrator. By looking at the error message looks like you don't have the write access to folders where Alteryx is installed and due to which it is not able to install the packages.

D3100
6 - Meteoroid
Thanks I have confirmed my company's app store has installed it with Admin user level. So I can't install the new packages. I do however have my own working python 3.7 virtual environments. Is there any way I can point Alteryx to use that environment instead? I found a config file in the Python_venv Tool folder called pyvenv.cfg. It looks like it has the python distribution address. But updating this file seems to have no effect on the Alteryx tool's python SDK. Is there such a file that can be updated to point Alteryx to a different Python base?
SydneyF
Alteryx Alumni (Retired)

Hi @D3100,

 

It is not possible to change the Python instance the Python tool is associated with. The Alteryx Engine is hardcoded to point to the environment shipped with the software. This is done to ensure that the ayx library has access to all of its dependencies. 

 

The developers of the Python tool are aware of the issue you are encountering and are working on a solution. Stay tuned.

 

- Sydney

 

vincekys
5 - Atom

Hi, i was thrown an error whilst executing Alteryx.write(df,1). Is there a solution for this?

 

Is Sci Dormant: Error: Unable to convert column from pandas type bool to Alteryx type Bool
Error: unable to write output table "data" (C:\Users\1609829\AppData\Local\Temp\1a964eb8b6ee63b2d5b818cbaa68aaab\1\output_1.yxdb)
ERROR: writing outgoing connection data 1
 
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-7-b5a401e02a88> in <module>
      1 # df = Alteryx.read('#1')
      2 
----> 3 Alteryx.write(df, 1)

c:\users\1609829\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\ayx\export.py in write(pandas_df, outgoing_connection_number, columns, debug, **kwargs)
     84     """
     85     return __CachedData__(debug=debug).write(
---> 86         pandas_df, outgoing_connection_number, columns=columns, **kwargs
     87     )
     88 

c:\users\1609829\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\ayx\CachedData.py in write(self, pandas_df, outgoing_connection_number, columns, output_filepath)
    571             try:
    572                 # get the data from the sql db (if only one table exists, no need to specify the table name)
--> 573                 data = db.writeData(pandas_df_out, "data", metadata=write_metadata)
    574                 # print success message
    575                 if outgoing_connection_number is not None:

c:\users\1609829\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\ayx\Datafiles.py in writeData(self, pandas_df, table, metadata)
   1129                             converting_to_lower_bit_size = int(
   1130                                 extractDigitsFromString(conversion_dtype)
-> 1131                             ) < int(extractDigitsFromString(dtype_str))   1132                             converting_from_float_to_nonfloat = (
   1133                                 dtype_str[:5] == "float"

ValueError: invalid literal for int() with base 10: ''
 
rd916
6 - Meteoroid

is there a better way to scrap information from a list of urls other than the python tool?  I am having an issue looping through the urls

D3100
6 - Meteoroid
Have you tried using Regex tool? Thats where I would look first.
rd916
6 - Meteoroid

@D3100 how do can you do this with regex? My quick few google searches have showed me to scrape data in python then use regex..

D3100
6 - Meteoroid
rd916, I'm not yet sure, but I'd be glad to give it a shot 😜 This list of URLs.. are you trying to check the websites themselves and then pull the data from those sites? That is what it sounds like. Assuming that is the case, can you provide a sample site your trying to pull from and what you're expecting to get out of it?
rd916
6 - Meteoroid

So this would be the a simple of the url list I am trying to scrape:

0                     http://seidenki-100.com/ads.txt
1                          http://oglasicg.me/ads.txt
2                    http://www.takepon.tokyo/ads.txt
3            http://gaihekitosounoarekore.com/ads.txt
4                     http://hochzeitswahn.de/ads.txt
5                       http://infosuroit.com/ads.txt
6                    http://tidewaternews.com/ads.txt
7                         http://hongkong.net/ads.txt
8                     http://theridesofar.com/ads.txt
9                      http://contattimsg.com/ads.txt
10                            http://fishc.ru/ads.txt

 I know all sites will not have a successful connection.  However, for the ones that do, they will a block of text within the body of the html per an industry standard

 

Thank you for the help, i have spent wayyy too much time on this.  I believe I am close on a solution, however, i am getting error because the function is trying to treat the record column as a url.

mc_wallendjack
8 - Asteroid

Excellent Tool Description. Very helpful!

Sunny_P
6 - Meteoroid

When the Python Tool operates, it seems to always ingest all the data before processing any of it (i.e. no batch processing). Since Python can handle this type of functionality with generators, can we update the tool so that it may do some preprocessing (like imports and data prep) and allow a defined generator function to be called repeatedly from a separate input handle and provide batch data frames on output for more parallel-like processing of data?

SydneyF
Alteryx Alumni (Retired)

Hi @Sunny_P,

 

You are correct, the way the Python tool currently functions is by reading in all of the input data at once and writing out all output data at once. Can you please post your idea for batch processing in the Python tool to our Ideas Forum? Once you do, please post back here with a link so other users can support your idea with likes. 

 

Thanks!

 

Sydney

Sunny_P
6 - Meteoroid

Thanks for that @SydneyF! I've posted the idea here: https://community.alteryx.com/t5/Alteryx-Designer-Ideas/Python-Tool-Parallel-Processing-Data/idi-p/4...

 

Please vote/comment for it!

AnanthS
5 - Atom

I am just starting with Alteryx Designer and would like to integrate with a python script I have written. I do not see the options within Alteryx for Python3 configuration. Here is the screen I have. Is there something I am missing in the installation.alteryx.JPG

 

Can someone guide me?

nishanttayal
6 - Meteoroid

@AnanthS 

 

As far as I know, currently there is no option to configure with existing Python but its automatically configured with the python environment with all necessary libraries installed. You can find Python tool under Developer section and you can execute your script in that tool.
Hope that helps!

 

Sunny_P
6 - Meteoroid

@AnanthS, Alteryx comes preinstalled with its own version of Python3 that integrates with Alteryx's own package library. There's no configuration with this possible. You'll just need to drag in the Python tool from the Developer toolset and paste your code into the Jupyter Notebook.

 

If you need to import other libraries, you can use the "ayx.Package.ImportPackage" function to pull it from PyPi. You can use the "ayx.Alteryx.read" and "ayx.Alteryx.write" functions for input and output for this tool's interactivity with other tools.

AnanthS
5 - Atom

Thanks Sunny and Nishant for your responses. Can you guide me where Developer toolset is? I am kind of lost navigating through the designer. When I click on any of the options, I am not being led to anything. Here is an example:

alteryx.JPG

 

How do I choose the input file?

 

 

 

Sunny_P
6 - Meteoroid

clipboard_image_0.png

The Python Tool. Open it up, and check the "configuration" for the Jupyter Notebook

 

 

Alternatively, if you're searching for any tool, resource, or any questions, just use the "Search bar" at the top right. You can even drag tools into your workflow!

 

EJMG
5 - Atom

Hi,

 

Will we be able to write out a plot soon?

 

Thanks!

smoskowitz
12 - Quasar

Hi All --

 

Ok, first off....new to Python so apologies for my lack of programming knowledge.....:(

 

The problem I am trying to solve is that there is an Oracle file that no longer conforms to standard XML and therefore throws errors in designer on the input tool or if running through the server. The suggestion from support was to import using the Python tool. 

 

So with much struggling and learning, I managed to make it work. below is my code:

 

from ayx import Alteryx
import pandas as pd
my_file = "C:\\Users\\seth.moskowitz\\Downloads\\Intercompany Report\\ic-all-actpub-Nov-2019-YTD.xlsx"
df = pd.read_excel(my_file)
Alteryx.write(df, 1)

 

My question is: Is there a way to make the my_file = to some variable that could come in via the directory tool or maybe through a file/browse? If so, would I have to break this apart to ad the double \ to the structure?

 

This report changes monthly and I wouldn't want to have to change that code

 

Otherwise, this does work and the data is brought in, now I need to make the file selection more dynamic and I am not quite sure how to do that. 

 

Thanks for your help. I look forward to learning more python.

 

Seth

Sunny_P
6 - Meteoroid

Seth,

 

The Python tool can accept multiple inputs, just make sure you name your connection and use it in the Python Tool, for example:

file_name = Alteryx.read("#2")

 If the connection coming in that has the file name is called "#2".

 

Finally, if that is coming in from an input, you should be fine with not adding additional backslashes to the string. You should be easily able to test this by writing this to the output.

 

Here's a minimal script you can put into your Python Tool:

 

from ayx import Package, Alteryx
import pandas as pd
file_name = Alteryx.read("#2")
df = pd.DataFrame({'File Name': [file_name['Field1'].values[0]]})
Alteryx.write(df, 1)

This takes in a Text Input with a single cell, with the value of "C:\Users\seth.moskowitz\Downloads\Intercompany Report\ic-all-actpub-Nov-2019-YTD.xlsx" (without quotes). You can feed it a filename too.

smoskowitz
12 - Quasar

@Sunny_P 

 

Thank you for your help. I believe I followed your instructions (except changing the #2 to a #1 and all that is output is the filename.

 

Below is the code I put in:

 

from ayx import Package, Alteryx
import pandas as pd
file_name = Alteryx.read("#1")
df = pd.DataFrame({'File Name': [file_name['Field1'].values[0]]})
Alteryx.write(df, 1)

My output 1 looks like this:

 

C:\Users\seth.moskowitz\Downloads\Intercompany Report\ic-all-actpub-Nov-2019-YTD.xlsx

I don't understand the line df = pd.Dataframe……..etc. enough to diagnose. What exactly is going on there?

 

Again, I appreciate your help.

 

Thanks,

Seth