Using Pandas and "to_sql" Runs VERY Slow in Alteryx Python Tool

[TallGibbs]

Hello Everyone,

I have a Jupyter Notebook that runs perfectly normal when running on my computer, outside of the Alteryx software.  Then, when I import the notebook and run it within Alteryx, it takes more than 5 minutes to run this 1 line of code:

"df_conversion_dict.to_sql(name=table_name, con=engine, schema=schema_name, if_exists='replace',index=False)"

That same line of code takes literally 2 seconds (wall time = 2s) when I run this outside of Alteryx.  Does anyone have any experience/ideas why trying to write a dataframe to SQL (connection to SSMS database) is running VERY slow through Alteryx software?  Both "Interactive" and "Production" modes very slow.  Production mode does run faster, but still takes about 4 minutes when my notebook on its own literally takes 2 seconds to execute that command.

Thank you for the help!

[acarter881]

Hey, @TallGibbs.

I have experience using the Python tool in Designer, but not doing what you're describing. In general, I prefer to use Visual Studio Code to run my Python code. Is it possible to run what you need in your Jupyter Notebook then read it into Designer?

I just think it has to do with how the Python tool is in Designer. I don't think it's optimized? I've always found it to lag quite a bit.

[geraldo]

@TallGibbs 

What I can tell you is what Alteryx is for if you want to use python.
do all your sql extractions via alteryx and what is really necessary that Alteryx does not serve you in python or R

[TallGibbs]

We are trying to build a fully automated workflow, and my current notebook is dependent on multiple processes before it.  The notebook would be "kicked off" once the precedening operations are completed inside Alteryx.  This why we are trying to integrate it into Alteryx.  We have no clue though why it is so much slower when ran inside Alteryx vs. running the notebook on its own is literally only a few seconds.

[TallGibbs]

Yeah, I understand your thought process but we are also organizationally limited by the number of licenses available for Alteryx.  So, I know python & R but I do not actually have the license to run Alteryx (or do any of my own testing in Atleryx).  So, I am trying to build as much of the workflow as I can using the tools available to me, and then integrate it at the end into Alteryx.