Double data type rounding to nearest whole number

[oracleoftemple]

I have a field that's a Double data type, and I'm applying a simple formula to it.  On the way into the Formula tool, all the values in that field are whole numbers (integers), although the data type is Double.  The formula, which is just a simple division, causes some of the values in that field to become fractional.  For instance, the value going in is 68, which is then divided by 6, so the answer is 11.3333333333.  The Data Preview box is showing the number as 11.333333333, but when the data comes out, it has been rounded to 11.  I checked the Cell Viewer to see if the .33333333 was still there, just not visible, but is shows 11.

If the data type is Double, shouldn't that allow digits after the decimal when a calculation results in a fraction?

[atcodedog05]

Hi @oracleoftemple 

Interestingly strange scenario are you facing the same issue with normal data the one which is not coming out of In-Db tool.

[Akash__on]

strange scenario .. i tried on small normal data but for me its coming as expected.

[atcodedog05 · Accepted solution]

Hi @oracleoftemple 

Create the calculation as a new column and try.

Hope this helps : )

[oracleoftemple]

This is strange.  Coming out of the In-DB tools, it shows up as a Double (file 1), and it also shows as Double in the Formula tool where I actually write the formula.  But when it comes out the Formula tool, it is somehow converted to an Integer (file 2).

[atcodedog05]

Happy to help : ) @oracleoftemple 

Cheers and have a nice day!