Designer taking over 20 hours to retrieve 3rd party data via ODBC

Everyone, I have an ODBC connector that is pulling roughly 4 million lines of data there is no workflow yet, just the data pull.

This pull is taking upwards of 21 hours... what steps can I take to speed this up?

Computer RAM?

Computer memory? currently at 50% remaining after data pull is Started.

increased Server capability? -- Do not have an Alteryx Server

Any help would be appreciated.

Workflow

Reporting

Tips and Tricks

Server

Engine

Accepted answers

All comments

Prometheus

@ThornhillTaylor Have you tried in-database tools?

ThornhillTaylor

@Prometheus, thanks for your comment. I tried the other day.. to no avail.. I am trying now, will update once the data has been pulled... currently at 3 minutes

ThornhillTaylor

that seems to have worked it took 4 minutes for 250k rows

CharlieS

also try specifying "with (nolock"/uncommitted read in case it's waiting on other processes.

caltang

Is this a recurring thing or a one off thing? The In-DB route would be a good pick as suggested by @Prometheus

ThornhillTaylor

This is a recurring situation... sorry my delay but the In-DB tools were taking excessive amounts of time to run... in fact 3 connections were over 18 hours with the largest of the 3 being 350,000 rows...

Im considering other options now as the ODBC connector my be my issue....

Any idea on writing python scripts to retrieve the data??

ThornhillTaylor

this is coming from Shopify... just no experience trying to do this

Quick Links

This months top contributors

atcodedog05 19598

Qiu 15879

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703