AlteryxGUI.exe exception when processing 400+ fields file

Yongcan

I'm comparing two files with 400+ fields using multiple fields formula.

FileA：

Field1,Field2...

Value1,Value2...

FileB:

Field1,Field2...

Value1,Value2...

Trying to the get result like below:

Field_Name,SAME,DIFF,EMPTY,SAME%,DIFF%,EMPTY%

Field1,X,Y,Z,a%,b%,c%

As lots fields involved, I'm editing XML directly.

The source file is only with 300 records for testing, but I notice the PC become slow ,(from task manager , AlteryxGUI.exe using 9G memory on the 16 Core, 32G work station.) and later start getting below exceptions.

AlteryxGui.exe - Exception

Breakpoint A breakpoint has bean reached.
2019-03-03 10_08_20-Window Breakpoint.png

Alteryx Desinger X64

An out of memory exception happened, the application may not be able e to recover.

Do you wan to exit?

2019-03-03 11_20_49-Window OOM.png

I'm using Alteryx Desinger X64 version 2018.4.4.54346. How can i trouble shoot such issue or any diagnose log etc?

Also i notice the Multiple-Field Formula's message/input/output is showing nothing though there are data flowing.

Error Message

Accepted answers

danilang

hi @Yongcan

While Alteryx can work with large numbers of columns it does start to slow down when you reach ~1000 fields. A lot of this has to do with the amount of metadata that Alteryx maintains about the datasets. You might want to consider transposing your data to bring it all into a single column and using a single formula tool.

This has the added benefits of 1. providing the output in field order and 2. Reducing the complexity of the overall workflow by having a formula tool that acts on a single column as opposed to a very large multi field formula.

I would still open a ticket with Alteryx support, since you shouldn't be getting the error message in the first place

Dan

Transpose.yxmd

All comments

danilang

hi @Yongcan

I would still open a ticket with Alteryx support, since you shouldn't be getting the error message in the first place

Dan

Transpose.yxmd

MarqueeCrew

@Yongcan ,

I'm in agreement with @danilang . I would most definitely transpose the data. I might alter the formula as:

[X] = [Y]

That will provide True or False without the IF statement (make the data type = Bool). ![SAME] works well for the DIFF field. I also might use something like:

IsEmpty(X) or IsEmpty(Y)

For the empty comparison.

Cheers,

Mark

Yongcan

Thanks, I believe this is the best practice for such case. I will try this solution later.

1st try, I implemented in urgly way. I was using "Multiple-Field Formula" with all fields name in the formula, that's cause the huge workflow flow file as it store metadata of each fields.

My second solution was use 400+ formula to compare the field. the code is similar and won't cause huge workflow file and get result i want.

Quick Links

This months top contributors

AkimasaKajitani 389

mceleavey 388

mbarone 337

Hollingsworth 335

LanisC 335