Caching ODBC query results

[fnb]

I have a module that queries a large amount of data from Redshift (~40 GB). It appears that the results are stored in memory until the query completes; consequently, my machine, which has 30 GB memory, crashes. This is a shame because Alteryx is good with maintaining memory <-> HDD balance.

Idea: Create a way to offload the query results onto the HDD as they are received.

[RyanNewsome]

Hi fnb, how is your workflow setup and what is your memory settings? Are you doing lots of sorting? I've pulled over 200gb from redshift on a machine with 64gb memory and everything worked fine.

http://help.alteryx.com/9.5/Memory_Utilization.htm

[fnb]

I am doing an in-DB query and then streaming out the records directly into a .yxdb file. The error I get is thus:

Data Stream Out (223) Error SQLExecute: [Amazon][RedShift ODBC] (30) Error occurred while trying to execute a query: out of memory for query result¶

[Community_Admin]

The status of this idea has been changed to 'Inactive'. This status indicates that:

1. The idea has not had activity in the form of likes or comments in over a year.

2. The idea has not reached ten likes.

3. The idea is still in the 'New Idea' status. 

However, this doesn't mean your idea won't be implemented! The Community can still like and comment on this idea. With enough renewed interest, this idea can be brought back into the 'New Idea' status. 

Thank you for contributing to the Alteryx Community and the Alteryx Product Idea Boards!