Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

Double data type rounding to nearest whole number

oracleoftemple
9 - Comet

I have a field that's a Double data type, and I'm applying a simple formula to it.  On the way into the Formula tool, all the values in that field are whole numbers (integers), although the data type is Double.  The formula, which is just a simple division, causes some of the values in that field to become fractional.  For instance, the value going in is 68, which is then divided by 6, so the answer is 11.3333333333.  The Data Preview box is showing the number as 11.333333333, but when the data comes out, it has been rounded to 11.  I checked the Cell Viewer to see if the .33333333 was still there, just not visible, but is shows 11.

 

If the data type is Double, shouldn't that allow digits after the decimal when a calculation results in a fraction?

5 REPLIES 5
atcodedog05
22 - Nova
22 - Nova

Hi @oracleoftemple 

 

Interestingly strange scenario are you facing the same issue with normal data the one which is not coming out of In-Db tool.

Akash__on
8 - Asteroid

strange scenario .. i tried on small normal data but for me its coming as expected.

Akash__on_0-1644422640962.png

 

atcodedog05
22 - Nova
22 - Nova

Hi @oracleoftemple 

 

Create the calculation as a new column and try.

 

Hope this helps : )

oracleoftemple
9 - Comet

This is strange.  Coming out of the In-DB tools, it shows up as a Double (file 1), and it also shows as Double in the Formula tool where I actually write the formula.  But when it comes out the Formula tool, it is somehow converted to an Integer (file 2).

atcodedog05
22 - Nova
22 - Nova

Happy to help : ) @oracleoftemple 

Cheers and have a nice day!

Labels