Start Free Trial

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

Formulas

qwerty
5 - Atom

I have set up this workflow just to calculate the difference between two numbers and for some reason, I am getting differences for some pairs of numbers but not for others. The numbers do equal each other. Below is a screenshot of the formula I'm using and the output I am getting:

qwerty_1-1631040421826.png

 

qwerty_0-1631040285625.png

 

2 REPLIES 2
phottovy
13 - Pulsar
13 - Pulsar

Hi @qwerty ,

 

I think your field 'Right_Sum_Effective_Amounts' must have some very small decimal values. -8.881e-16 is the equivalent of -0.0000000000000008881. If you change your formula to 

 

[Sum_Effective_Amounts] - Round([Right_Sum_Effective_Amounts], 0.01) * 100

 

you should get equal values.

mst3k
11 - Bolide

it has to do with how the computer stores floating numbers

 

https://docs.microsoft.com/en-us/cpp/build/why-floating-point-numbers-may-lose-precision?view=msvc-1...

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

 

you'll probably notice the same thing happen in excel from time to time.

in the case of alteryx, you can put a round() function around your values, or change the datatype to a specific precision/scale numeric

Labels
Top Solution Authors