Why am i getting different results doing almost the same thing?

[Voska]

Hello,

Can someone please explain why Alteryx math seems off?

[jrgo]

@Voska,

My guess is that it's because you're not including the additional precision your [calc] expression is generating. The result preview is rounding for clarity.

[Voska]

I don't think i am actual familiar with what you described, can you provide an example of precision setting you are referring to?  Thank you!

[MSalvage]

@Voska,

Pretty sure it is because Alteryx visually rounds to 6 decimal places. Meaning it shows 0.032877, but the true value retained in the [calc] might be 0.032876711038961038961038961033896 = 10.126027/308.

Hope this makes sense,

MSalvage

[jrgo · Accepted solution]

@Voska,

Illustration of decimal precision 

Hope this helps!