I'm comparing two files with 400+ fields using multiple fields formula.
FileA:
Field1,Field2...
Value1,Value2...
FileB:
Field1,Field2...
Value1,Value2...
Trying to the get result like below:
Field_Name,SAME,DIFF,EMPTY,SAME%,DIFF%,EMPTY%
Field1,X,Y,Z,a%,b%,c%
As lots fields involved, I'm editing XML directly.
The source file is only with 300 records for testing, but I notice the PC become slow ,(from task manager , AlteryxGUI.exe using 9G memory on the 16 Core, 32G work station.) and later start getting below exceptions.
AlteryxGui.exe - Exception
Breakpoint A breakpoint has bean reached.
Alteryx Desinger X64
An out of memory exception happened, the application may not be able e to recover.
Do you wan to exit?
I'm using Alteryx Desinger X64 version 2018.4.4.54346. How can i trouble shoot such issue or any diagnose log etc?
Also i notice the Multiple-Field Formula's message/input/output is showing nothing though there are data flowing.
Solved! Go to Solution.
hi @Yongcan
While Alteryx can work with large numbers of columns it does start to slow down when you reach ~1000 fields. A lot of this has to do with the amount of metadata that Alteryx maintains about the datasets. You might want to consider transposing your data to bring it all into a single column and using a single formula tool.
This has the added benefits of 1. providing the output in field order and 2. Reducing the complexity of the overall workflow by having a formula tool that acts on a single column as opposed to a very large multi field formula.
I would still open a ticket with Alteryx support, since you shouldn't be getting the error message in the first place
Dan
@Yongcan ,
I'm in agreement with @danilang . I would most definitely transpose the data. I might alter the formula as:
[X] = [Y]
That will provide True or False without the IF statement (make the data type = Bool). ![SAME] works well for the DIFF field. I also might use something like:
IsEmpty(X) or IsEmpty(Y)
For the empty comparison.
Cheers,
Mark
Thanks, I believe this is the best practice for such case. I will try this solution later.
1st try, I implemented in urgly way. I was using "Multiple-Field Formula" with all fields name in the formula, that's cause the huge workflow flow file as it store metadata of each fields.
My second solution was use 400+ formula to compare the field. the code is similar and won't cause huge workflow file and get result i want.