Bring your best ideas to the AI Use Case Contest! Enter to win 40 hours of expert engineering support and bring your vision to life using the powerful combination of Alteryx + AI. Learn more now, or go straight to the submission form.
Start Free Trial

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

Formulas

qwerty
5 - Atom

I have set up this workflow just to calculate the difference between two numbers and for some reason, I am getting differences for some pairs of numbers but not for others. The numbers do equal each other. Below is a screenshot of the formula I'm using and the output I am getting:

qwerty_1-1631040421826.png

 

qwerty_0-1631040285625.png

 

2 REPLIES 2
phottovy
13 - Pulsar
13 - Pulsar

Hi @qwerty ,

 

I think your field 'Right_Sum_Effective_Amounts' must have some very small decimal values. -8.881e-16 is the equivalent of -0.0000000000000008881. If you change your formula to 

 

[Sum_Effective_Amounts] - Round([Right_Sum_Effective_Amounts], 0.01) * 100

 

you should get equal values.

mst3k
11 - Bolide

it has to do with how the computer stores floating numbers

 

https://docs.microsoft.com/en-us/cpp/build/why-floating-point-numbers-may-lose-precision?view=msvc-1...

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

 

you'll probably notice the same thing happen in excel from time to time.

in the case of alteryx, you can put a round() function around your values, or change the datatype to a specific precision/scale numeric

Labels
Top Solution Authors