Bring your best ideas to the AI Use Case Contest! Enter to win 40 hours of expert engineering support and bring your vision to life using the powerful combination of Alteryx + AI. Learn more now, or go straight to the submission form.
Start Free Trial

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

Rounding Causing Output Issues

TJBell
5 - Atom

I am trying to add a number of different columns together but my output is off from what appears to be a rounding issue. I have all the columns rounded to four decimal places, but when I look at the generated output vs. expected output I am off by over 4 tenths. When I manually calculate the figures on the output I get the correct number, but it is not correct in the Alteryx output. Is there anything I am missing in order to get the Alteryx output to match the manual calculation? 

 

TJBell_0-1620409964997.png

 

 

4 REPLIES 4
YEM
8 - Asteroid

We need more information to help you.

 

Please post a yxzp of your workflow

 

OR

 

Post screenshots
1) a select tool config with the data types of the 4 columns

2) formula tool config where you are doing the math

TJBell
5 - Atom

I put together the attached workflow which has the correct output in NAV difference. Where I am running into an issue is with a larger workflow that is calculating all of the existing columns. For some reason when I do not hardcode the numbers into designer I run into the rounding issue. 

KarolinaRoza
11 - Bolide

hi @TJBell 

 

I am not sure how you got NAV = 31.4337 in your offline spreadsheet. 

Looking at values and formulas it doesn't sum up to this amount.

I think Alteryx calculate it correctly.

 

Karolina

danilang
19 - Altair
19 - Altair

Hi @TJBell 

 

In the example you attached, the difference is 0.0001 in Alteryx.  This matches the difference if you do the same calculation in Excel(attached)

 

danilang_0-1620475043038.png

 

In your original post, you mention that the difference is off by .4.  Can you post the original data?

 

Dan

 

 

 

Labels
Top Solution Authors