I have a field that's a Double data type, and I'm applying a simple formula to it. On the way into the Formula tool, all the values in that field are whole numbers (integers), although the data type is Double. The formula, which is just a simple division, causes some of the values in that field to become fractional. For instance, the value going in is 68, which is then divided by 6, so the answer is 11.3333333333. The Data Preview box is showing the number as 11.333333333, but when the data comes out, it has been rounded to 11. I checked the Cell Viewer to see if the .33333333 was still there, just not visible, but is shows 11.
If the data type is Double, shouldn't that allow digits after the decimal when a calculation results in a fraction?
Solved! Go to Solution.
Interestingly strange scenario are you facing the same issue with normal data the one which is not coming out of In-Db tool.
strange scenario .. i tried on small normal data but for me its coming as expected.
Happy to help : ) @oracleoftemple
Cheers and have a nice day!