Using Pandas and "to_sql" Runs VERY Slow in Alteryx Python Tool

Question

Hello Everyone,

I have a Jupyter Notebook that runs perfectly normal when running on my computer, outside of the Alteryx software.  Then, when I import the notebook and run it within Alteryx, it takes more than 5 minutes to run this 1 line of code:

"df_conversion_dict.to_sql(name=table_name, con=engine, schema=schema_name, if_exists='replace',index=False)"

That same line of code takes literally 2 seconds (wall time = 2s) when I run this outside of Alteryx.  Does anyone have any experience/ideas why trying to write a dataframe to SQL (connection to SSMS database) is running VERY slow through Alteryx software?  Both "Interactive" and "Production" modes very slow.  Production mode does run faster, but still takes about 4 minutes when my notebook on its own literally takes 2 seconds to execute that command.

Thank you for the help!

TallGibbs · Answer

Yeah, I understand your thought process but we are also organizationally limited by the number of licenses available for Alteryx.  So, I know python & R but I do not actually have the license to run Alteryx (or do any of my own testing in Atleryx).  So, I am trying to build as much of the workflow as I can using the tools available to me, and then integrate it at the end into Alteryx.

TallGibbs · Answer

We are trying to build a fully automated workflow, and my current notebook is dependent on multiple processes before it.  The notebook would be "kicked off" once the precedening operations are completed inside Alteryx.  This why we are trying to integrate it into Alteryx.  We have no clue though why it is so much slower when ran inside Alteryx vs. running the notebook on its own is literally only a few seconds.

geraldo · Answer

@TallGibbs

What I can tell you is what Alteryx is for if you want to use python.
do all your sql extractions via alteryx and what is really necessary that Alteryx does not serve you in python or R