Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

Read database records in sets (1000's)

pawan_bagdiya
7 - Meteor

Hello, 

 

I want to read a database table which has millions of records. I am trying to read that tables in batches (set of 1000) so that I can avoid physical memory error. Here is the image of the workflow. 

 

1. Input data (retrieving the data from database table having high volume of rows)

2. Output will be again writing to different database 

 

I want to build a workflow to read records in 1000's , add/update into different database  and loop until all records gets updated.  Can you share any sample workflow? 

 

pawan_bagdiya_0-1607980889704.png

 

1 REPLY 1
Qiu
20 - Arcturus
20 - Arcturus

@pawan_bagdiya 
Do you data have a Primary key, so we can do something like below then update with a Batch Macro?

ELECT top 10000 ... 
WHERE id > @maxid 
ORDER BY id

bel 

Labels