Azure SQL DB Connection: Other than ODBC

Riggsy

Hi There,

In the past I have used ODBC to connect to our Azure SQL DB's. While this was viable in the past with small amounts of data (Few million records), I have ran into challenges with bigger datasets, specifically the Data Stream Out tool. Currently it takes ~13 hours to stream out ~117m records into the canvas.

Are there better options I can use to connect Alteryx to an Azure SQL DB, to stream data to the canvas? I understand there are In-DB tools, however this severely limits the capabilities of Alteryx Designer.

At this point we have had to move away from utilizing Alteryx for big datasets as it has severely hindered our processing times.

Any help or best practices would be much appreciated! Thank you!

apathetichell

Taking a step back - you are asking if these is a faster way than an ODBC driver to bring 117mm rows of data from a SQL DB onto your personal laptop.The short is - probably not - but before we get there the key question is can you increase the ram size of your machine and monitor execution time? are you using amp (turn it off), and can you also turn off other memory intensive programs (ie Chrome/Excel) while running your workflow? these are your key factors - but you're really limited by network traffic speeds and to a much larger extent system resources. 

Is there another way to do this - maybe run your query in Azure/export to a blob storage/download your file as csv and pick up for processing? Does that sound like a pain - it is.  I'd strongly recommend doing as much as you can In-DB and only bring what you need on canvas.