Extracting records from database using dynamic input

[vrbravo]

I have like 100,000 records in an excel file containing employee ID.  i am trying to get salaryf from the Table_Emp for all records. So i am using DYNAMIC INPUT Tool and replace the WHERE clause with the emp_id for each row..

Select salary from Table_Emp Where Emp_ID = '100' 

The problem is i am getting memory exception error while the workflow is running .

Is there any alternate way to extract data from database for large inputs  ?

Thank you.

[rkapoor]

@vrbravo have you tried to use the in-db tools?

[vrbravo]

No sir, i have not before. would a dynamic input and IN DB dynamic input has different process method for large data set ?

[rkapoor]

@vrbravo the in-db will be a lot better for you with large data sets and volumes because it leverages the power of the db server. Give this a try and let me know how you get on

[rkapoor]

@vrbravo have a look at this link to give you an intro into in-db tools:

https://community.alteryx.com/t5/Alteryx-Designer-Knowledge-Base/FAQ-How-Do-the-In-Database-tools-Wo...

[vrbravo]

the database we have moved now is DB2 for which the in-DB tool doesnt support. is there any alternate workaround ?

[rkapoor]

@vrbravo you may want to play with the memory settings on your Designer maybe.

https://help.alteryx.com/current/designer/memory-use.

Also, check to see if the exceptions aren't on the db side maybe?