Python Tool Timeouts When Running Workflow

[rdoptis]

Hi There,

Timeouts can occur when executing a workflow that uses the Python tool if any of the notebook cells contain intensive operations requiring 31+ seconds of execution time.

The error message shown in the Alteryx message log: 

Error: Blobby Output (9): Tool #2: [NbConvertApp] Converting notebook C:\ProgramData\Alteryx\Engine\96e4c0ab-5610-4858-80e0-3625bb421cec\20\workbook.ipynb to html
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] ERROR | Timeout waiting for execute reply (30s).
Traceback (most recent call last):
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 324, in _wait_for_reply
    msg = self.kc.shell_channel.get_msg(timeout=timeout)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\jupyter_client\blocking\channels.py", line 57, in get_msg
    raise Empty
queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Ryan\AppData\Local\Alteryx\bin\Miniconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\Ryan\AppData\Local\Alteryx\bin\Miniconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Ryan\AppData\Local\Alteryx\bin\Miniconda3\PythonTool_Venv\Scripts\jupyter-nbconvert.EXE\__main__.py", line 9, in <module>
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\jupyter_core\application.py", line 266, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\traitlets\config\application.py", line 658, in launch_instance
    app.start()
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 325, in start
    self.convert_notebooks()
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 493, in convert_notebooks
    self.convert_single_notebook(notebook_filename)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 464, in convert_single_notebook
    output, resources = self.export_single_notebook(notebook_filename, resources, input_buffer=input_buffer)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\nbconvertapp.py", line 393, in export_single_notebook
    output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 174, in from_filename
    return self.from_file(f, resources=resources, **kw)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 192, in from_file
    return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\html.py", line 85, in from_notebook_node
    return super(HTMLExporter, self).from_notebook_node(nb, resources, **kw)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\templateexporter.py", line 280, in from_notebook_node
    nb_copy, resources = super(TemplateExporter, self).from_notebook_node(nb, resources, **kw)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 134, in from_notebook_node
    nb_copy, resources = self._preprocess(nb_copy, resources)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\exporters\exporter.py", line 311, in _preprocess
    nbc, resc = preprocessor(nbc, resc)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\base.py", line 47, in __call__
    return self.preprocess(nb, resources)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 262, in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\base.py", line 69, in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 280, in preprocess_cell
    reply, outputs = self.run_cell(cell, cell_index)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 348, in run_cell
    exec_reply = self._wait_for_reply(msg_id, cell)
  File "c:\users\ryan\appdata\local\alteryx\bin\miniconda3\pythontool_venv\lib\site-packages\nbconvert\preprocessors\execute.py", line 337, in _wait_for_reply
    raise exception("Cell execution timed out")
TimeoutError: Cell execution timed out

Does anyone have any creative ideas on how to adjust the timeout or eliminate it? Nbconvert has parameters for these timeouts, however I'm unsure how they can be adjusted when Alteryx Designer orchestrates the process...

Thanks,

Ryan

[patrick_digan · Accepted solution]

@rdoptis There is a post over here where @PaulN provided a good solution. For myself, I just modified this file from 30 to -1 on line 68 of the code: 

"C:\Program Files\Alteryx\bin\Miniconda3\PythonTool_venv\Lib\site-packages\nbconvert\preprocessors\execute.py"

[rdoptis]

Thanks @patrick_digan! My search skills must be atrophying as I did search to see if anyone else posted about this.

This solution works as a quick fix and I will make the same change. However, this solution can be problematic if I share the workflow with a colleague, publish it to our private gallery, or upgrade Alteryx (i'm assuming these files would get replaced on reinstall). 

Hopefully the Alteryx team can officially support the parameterization of this going forward.

Best regards,

Ryan

[patrick_digan]

@rdoptis I'm definitely still searching for a long term enterprise solution as well.

[OndrejC]

Yes would be great if we have long term enterprise solution for this.

[HeatherMHarris]

Thank you, @patrick_digan. The fix by @PaulN did not work for me.

Editing the execute.py to have a timeout of -1 instead of 30 does work, BUT... I had to trick my machine to let me "edit" that file, as IT locks down access on machines. I copied and edited the file in a directory where I have access permissions, and then did a copy and paste back into the directory. Edit and save wouldn't work but copying and replace did.

Thank you!

Heather

@HeatherMHarris

[PaulN]

Thanks for sharing @HeatherMHarris!

Bear in mind that these modifications may be lost if Designer has to be reinstalled or upgraded.

Paul Noirel

Sr Customer Support Engineer, Alteryx

[HeatherMHarris]

Thanks for that reminder, @PaulN. It would be great if it the ability to configure this via the Python Tool would be in the next release... :-)

Heather

[gsjacobs]

Thank you for this post. It fixed the issue, even if it's temporary, this is great news.