AlteryxGUI.exe exception when processing 400+ fields file

[Yongcan]

I'm comparing two files with 400+ fields using multiple fields formula.

FileA：

Field1,Field2...

Value1,Value2...

FileB:

Field1,Field2...

Value1,Value2...

Trying to the get result like below:

Field_Name,SAME,DIFF,EMPTY,SAME%,DIFF%,EMPTY%

Field1,X,Y,Z,a%,b%,c%

As lots fields involved, I'm editing XML directly. 

The source file is only with 300 records for testing, but I notice the PC become slow ,(from task manager , AlteryxGUI.exe using 9G memory on the 16 Core, 32G work station.) and later start getting below exceptions.

AlteryxGui.exe - Exception

Breakpoint A breakpoint has bean reached.

Alteryx Desinger X64

An out of memory exception happened, the application may not be able e to recover.

Do you wan to exit?

I'm using Alteryx Desinger X64 version 2018.4.4.54346. How can i trouble shoot such issue or any diagnose log etc?

Also i notice the Multiple-Field Formula's message/input/output is showing nothing though there are data flowing.

[danilang · Accepted solution]

hi @Yongcan 

While Alteryx can work with large numbers of columns it does start to slow down when you reach ~1000 fields.  A lot of this has to do with the amount of metadata that Alteryx maintains about the datasets.  You might want to consider transposing your data to bring it all into a single column and using a single formula tool.

This has the added benefits of 1. providing the output in field order and 2.  Reducing the complexity of the overall workflow by having a formula tool that acts on a single column as opposed to a very large multi field formula.

I would still open a ticket with Alteryx support, since you shouldn't be getting the error message in the first place 

Dan 

[MarqueeCrew]

@Yongcan ,

I'm in agreement with @danilang .  I would most definitely transpose the data.  I might alter the formula as:

[X] = [Y]

That will provide True or False without the IF statement (make the data type = Bool).  ![SAME] works well for the DIFF field.  I also might use something like:

IsEmpty(X) or IsEmpty(Y)

For the empty comparison.

Cheers,

Mark

[Yongcan]

Thanks, I believe this is the best practice for such case. I will try this solution later.

1st try, I implemented in urgly way. I was using "Multiple-Field Formula" with all fields name in the formula, that's cause the huge workflow flow file as it store metadata of each fields.

My second solution was use 400+ formula to compare the field. the code is similar and won't cause huge workflow file and get result i want.