Bring your best ideas to the AI Use Case Contest! Enter to win 40 hours of expert engineering support and bring your vision to life using the powerful combination of Alteryx + AI. Learn more now, or go straight to the submission form.
Start Free Trial

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

Upload data in SQL in batches

Designer07
5 - Atom

I am trying to copy a table from one SQL server to another. The data size is huge 3M+ rows, it will take almost 6+ hours to copy this . The workflow ends abruptly sometimes at 200k rows, 500k or 1M rows whenever my system sleeps, wifi connection issue or anything else. Is there an alternative to copy data from one SQL to another ? Anything that can be done to expedite. Please note I'm very new to Alteryx. Have tried SQL import export wizard as well but that too stops in between. I want a solution within alteryx

3 REPLIES 3
caltang
17 - Castor
17 - Castor

Errr.... I would suggest for you to really understand what you're trying to do first. If you're trying to move from one SQL Server to another, I do not think Alteryx is the best tool -> https://www.vinchin.com/en/blog/sql-server-database-move-to-another-server.html

 

But if you insist on Alteryx, then you will need to break your data into chunks. 3M+ rows is okay, how many fields are we talking about here? The larger your frame, the slower it is. 

 

Refer below: 

https://knowledge.alteryx.com/index/s/article/How-To-Migrate-Entire-Databases-Using-Alteryx-15834606...

 

EDIT: Here's something relevant for consideration: https://community.alteryx.com/t5/Alteryx-Connect-Discussions/Migrating-Data-Sources/td-p/443175 

Calvin Tang
Alteryx ACE
https://www.linkedin.com/in/calvintangkw/
Designer07
5 - Atom

There are total 75 Columns. How do I upload in chunks ? Create separate workflows ?

caltang
17 - Castor
17 - Castor

Is there a dimension which you reference the split? For example, if your data is granular, you can split it by Month-Year. Or by product... etc.

 

What sort of data are you working with? 

 

The back-up of back-ups would be to stream the data out as YXDBs, then stream the data back via write access in to your 2nd SQL Server.

Calvin Tang
Alteryx ACE
https://www.linkedin.com/in/calvintangkw/
Labels
Top Solution Authors