I am trying to add a number of different columns together but my output is off from what appears to be a rounding issue. I have all the columns rounded to four decimal places, but when I look at the generated output vs. expected output I am off by over 4 tenths. When I manually calculate the figures on the output I get the correct number, but it is not correct in the Alteryx output. Is there anything I am missing in order to get the Alteryx output to match the manual calculation?
We need more information to help you.
Please post a yxzp of your workflow
OR
Post screenshots
1) a select tool config with the data types of the 4 columns
2) formula tool config where you are doing the math
I put together the attached workflow which has the correct output in NAV difference. Where I am running into an issue is with a larger workflow that is calculating all of the existing columns. For some reason when I do not hardcode the numbers into designer I run into the rounding issue.
hi @TJBell
I am not sure how you got NAV = 31.4337 in your offline spreadsheet.
Looking at values and formulas it doesn't sum up to this amount.
I think Alteryx calculate it correctly.
Karolina
Hi @TJBell
In the example you attached, the difference is 0.0001 in Alteryx. This matches the difference if you do the same calculation in Excel(attached)
In your original post, you mention that the difference is off by .4. Can you post the original data?
Dan