Formulas

Question

I have set up this workflow just to calculate the difference between two numbers and for some reason, I am getting differences for some pairs of numbers but not for others. The numbers do equal each other. Below is a screenshot of the formula I'm using and the output I am getting:

mst3k · Answer

it has to do with how the computer stores floating numbers

https://docs.microsoft.com/en-us/cpp/build/why-floating-point-numbers-may-lose-precision?view=msvc-160

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

you'll probably notice the same thing happen in excel from time to time.

in the case of alteryx, you can put a round() function around your values, or change the datatype to a specific precision/scale numeric

phottovy · Answer

Hi @qwerty ,

I think your field 'Right_Sum_Effective_Amounts' must have some very small decimal values. -8.881e-16 is the equivalent of -0.0000000000000008881. If you change your formula to

[Sum_Effective_Amounts] - Round([Right_Sum_Effective_Amounts], 0.01) * 100

you should get equal values.