Morning everyone, I have a big delivery tomorrow and this is driving me crazy. Lets say that im using a formula calculation:
Score (preview result) 0.615345
[field1]*[field2]
Fixed Decimal 19.6
but once is printed the column score shows:
0.619570
Same format Fixed Decimal 19.6
BTW if I perform this calculationin excel I get the 0.615345
Please HELP!
Solved! Go to Solution.
Could you provide the data or workflow ?
I wish I could, but my company laptops is very restricted, can’t upload anything. Not even screenshots.
Since this change is too large to be attributed to a rounding error, trace the value through all the various tools between the formula and your final output, starting at the output anchor of the formula tool.
If the value is different at the output anchor of the same formula tool, then there is an error in the preview. These are very rare but I have seen them before.
Check the value at every tool in the chain and look for some place where it might change.
If the error is only occurring when the workflow is run, add in some debugging information. Connect the output of the formula directly to an output tool that writes to a yxdb file or a csv. After run, open this file and look at not only the formula field but also the two fields that are multiplied to give the results. The input values might not be what you expect
Dan
Found the error, one of my element being multiplied was rounded to 2 decimal places. This was causing the variation on the printed column, once I change that to 6 decimal places the numbers are matching. Thanks Dan!
User | Count |
---|---|
19 | |
15 | |
13 | |
9 | |
8 |