I have like 100,000 records in an excel file containing employee ID. i am trying to get salaryf from the Table_Emp for all records. So i am using DYNAMIC INPUT Tool and replace the WHERE clause with the emp_id for each row..
Select salary from Table_Emp Where Emp_ID = '100'
The problem is i am getting memory exception error while the workflow is running .
Is there any alternate way to extract data from database for large inputs ?
Thank you.
@vrbravo have you tried to use the in-db tools?
No sir, i have not before. would a dynamic input and IN DB dynamic input has different process method for large data set ?
@vrbravo the in-db will be a lot better for you with large data sets and volumes because it leverages the power of the db server. Give this a try and let me know how you get on
@vrbravo have a look at this link to give you an intro into in-db tools:
https://community.alteryx.com/t5/Alteryx-Designer-Knowledge-Base/FAQ-How-Do-the-In-Database-tools-Wo...
the database we have moved now is DB2 for which the in-DB tool doesnt support. is there any alternate workaround ?
@vrbravo you may want to play with the memory settings on your Designer maybe.
https://help.alteryx.com/current/designer/memory-use.
Also, check to see if the exceptions aren't on the db side maybe?