Memory Allocation error writing to Tableau hyper

[MorganG]

Has anyone experienced this memory allocation issue. It looks like the hyperd.exe is lost.

I have been reading the posts and have tried everything that I could find. My next step is to install the 2019.3 nonAdmin version that my colleagues use successfully.

Situation: Large data writes to Tableau hyper files result in 

"Error: Output Data (139): Cannot allocate 953942016 bytes of memory: exceeding allocation limit of 54002185012 bytes for global memory limit"

My observation is that Alteryx successfully output the yxdb (was at 100%) and the hyper outputfile showed 100% but the workflow was still running before the error was thrown.

Azure Virtual Machine: 

RAM 64GB

Disk 4TB

Alteryx Designer Admin: 2019.4.8.2207

Runs on colleagues machine with Alteryx Designer Admin: 2019.3,  less memory and output to network drive.

This is now a simple Input tool / output tool workflow

Variations that include  a select tool to change string to Wstring and a cleanse Tool.

I tired the following 

- Set Alteryx User Default Temporary Directory to 😧 drive

- Change Browse Everywhere Settings Memory Limit per Anchor from 1024 to 2048

- Changed Join/Sort Memory from 16383 to 32766MB caculated to 50% of memory

- RESULT: Error: Output Data (139): Cannot allocate 953942016 bytes of memory: exceeding allocation limit of 54002185012 bytes for global memory limit"

- Changed Join/Sort Memory from  32766 to 16383MB

- Changed Browse Everywhere Settings Memory Limit per Anchor to 1024

- Changed user memory to 8154

- RESULT: Error: Output Data (1): Cannot allocate 1620705280 bytes of memory: exceeding allocation limit of 54116211098 bytes for global memory limit

- Changed user memory to 2000

- Changed Join/Sort Memory to 4096

- Changed all string to Wstring and all v_string to v_Wstring

- RESULT: Error: Output Data (1): Cannot allocate 1620705280 bytes of memory: exceeding allocation limit of 54116211098 bytes for global memory limit

Current Activity:

Uninstall Alteryx Admin 2019.4 

Install Alteryx non-Admin 2019.3

[BrandonB]

Have you also tried an autofield before output? Sometimes this can also help. 

https://help.alteryx.com/current/AutoField.htm

[MorganG]

Hi Brandon,

The workflow scenarios. 

The one not shown was with just input -->autofield-->output.

And the last where the output was to a .tde that completed successfully.

it is interesting that the autofield actually changes some of the V_WString fields from the select tool back to V_String (Info: Auto Field (19): The FieldType of "My Adult Cat" changed to: V_String(20).

 

Morgan

[MorganG · Accepted solution]

The solution that I came up with was to break the creation of the .hyper file into 3 separate chunks.

This is really a work around because it runs in half the time on a colleagues machine.