community
cancel
Showing results for 
Search instead for 
Did you mean: 

Alteryx designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.
#SANTALYTICS

Gather all 9 clues to complete the final Weekly Challenge on Dec 16!

Learn More
SOLVED

Bulk data loading from SQLServer to Greenplum

Atom

 

 

Need an advise here,

i'm pulling a large dataset from SQLServer (10M rows) and need to move into Greenplum DB

I tried both with connecting using Input Data (SQLServer) and Output Data (GP) and also Connect In-DB (SQLServer) and Write Data In-DB (GP)

Any approach is taking a life to complete at the point that i have to cancel the process (to give an idea, over the weekend it ran for 18hours and advanced no further than 1%)

 

Any good advice or trick to speed up these sort of massive bulk data loading would be very very highly appreciated!

 

thanks in advance,

Regards,

Erick

Highlighted
Community Operations Manager
Community Operations Manager

@eamador,

 

If you are just moving data from SQL to Greenplum, Alteryx really is not the tool to just move data from one database to another. Alteryx brings in the data and creates temporary files to process the changes with your data. This is why you can play in Alteryx and never make changes to the original data. Alteryx is also a memory intensive tool and if your machine does not have the memory capacity to handle large amounts of memory use then you will experience slowness. I would imagine if you checked how your memory availability was on your machine you would most likely see it pegged to 100% when running that workflow. So if you are just attempting to move that data and not doing any changes to it you are essentially moving it twice.

 

You can find technical specifications here

 

DanM

Labels