Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

Rounding Causing Output Issues

TJBell
5 - Atom

I am trying to add a number of different columns together but my output is off from what appears to be a rounding issue. I have all the columns rounded to four decimal places, but when I look at the generated output vs. expected output I am off by over 4 tenths. When I manually calculate the figures on the output I get the correct number, but it is not correct in the Alteryx output. Is there anything I am missing in order to get the Alteryx output to match the manual calculation? 

 

TJBell_0-1620409964997.png

 

 

4 REPLIES 4
YEM
8 - Asteroid

We need more information to help you.

 

Please post a yxzp of your workflow

 

OR

 

Post screenshots
1) a select tool config with the data types of the 4 columns

2) formula tool config where you are doing the math

TJBell
5 - Atom

I put together the attached workflow which has the correct output in NAV difference. Where I am running into an issue is with a larger workflow that is calculating all of the existing columns. For some reason when I do not hardcode the numbers into designer I run into the rounding issue. 

KarolinaRoza
11 - Bolide

hi @TJBell 

 

I am not sure how you got NAV = 31.4337 in your offline spreadsheet. 

Looking at values and formulas it doesn't sum up to this amount.

I think Alteryx calculate it correctly.

 

Karolina

danilang
19 - Altair
19 - Altair

Hi @TJBell 

 

In the example you attached, the difference is 0.0001 in Alteryx.  This matches the difference if you do the same calculation in Excel(attached)

 

danilang_0-1620475043038.png

 

In your original post, you mention that the difference is off by .4.  Can you post the original data?

 

Dan

 

 

 

Labels