Let’s talk Alteryx Copilot. Join the live AMA event to connect with the Alteryx team, ask questions, and hear how others are exploring what Copilot can do. Have Copilot questions? Ask here!
Start Free Trial

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

Read database records in sets (1000's)

pawan_bagdiya
7 - Meteor

Hello, 

 

I want to read a database table which has millions of records. I am trying to read that tables in batches (set of 1000) so that I can avoid physical memory error. Here is the image of the workflow. 

 

1. Input data (retrieving the data from database table having high volume of rows)

2. Output will be again writing to different database 

 

I want to build a workflow to read records in 1000's , add/update into different database  and loop until all records gets updated.  Can you share any sample workflow? 

 

pawan_bagdiya_0-1607980889704.png

 

1 REPLY 1
Qiu
21 - Polaris
21 - Polaris

@pawan_bagdiya 
Do you data have a Primary key, so we can do something like below then update with a Batch Macro?

ELECT top 10000 ... 
WHERE id > @maxid 
ORDER BY id

bel 

Labels
Top Solution Authors