Rounding Causing Output Issues

Question

I am trying to add a number of different columns together but my output is off from what appears to be a rounding issue. I have all the columns rounded to four decimal places, but when I look at the generated output vs. expected output I am off by over 4 tenths. When I manually calculate the figures on the output I get the correct number, but it is not correct in the Alteryx output. Is there anything I am missing in order to get the Alteryx output to match the manual calculation?

danilang · Answer

Hi @TJBell

In the example you attached, the difference is 0.0001 in Alteryx.  This matches the difference if you do the same calculation in Excel(attached)

In your original post, you mention that the difference is off by .4.  Can you post the original data?

Dan

Nav.xlsx

KarolinaRoza · Answer

hi @TJBell

I am not sure how you got NAV = 31.4337 in your offline spreadsheet.

Looking at values and formulas it doesn't sum up to this amount.

I think Alteryx calculate it correctly.

Karolina

TJBell · Answer

I put together the attached workflow which has the correct output in NAV difference. Where I am running into an issue is with a larger workflow that is calculating all of the existing columns. For some reason when I do not hardcode the numbers into designer I run into the rounding issue.

Rounding Example.yxmd