cancel
Showing results for
Did you mean:

# Alteryx designer Discussions

SOLVED

## Rounding problems

Meteoroid

Hi All,

the rounding tool doesn't seem to be rounding in the conventional way

i used round([year 2 avg],0.1) this is fine for all except row 2. Is there any round this?

Alteryx Partner

Hi @SAPaul,

The reason for this is there can be slight variations in the way computers store these double data types due to binary conversions.

From an earlier post i've read previously (https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Alteryx-Odd-Behavior/td-p/89086)

For example:

206.95 when stored as a double in binary is actually '206.94999999999998863131622783839702606201171875' and when rounded at the first decimal place becomes 206.9

1.45 when stored as a double is actually '1.4499999999999999555910790149937383830547332763671875' which when rounded to 1dp becomes 1.4

1.55 when stored as a double is '1.5500000000000000444089209850062616169452667236328125' which rounds to 1.6 at 1dp

Taking a look at an example we see the true way these numbers are being stored:

The way round this would be to round to 1 extra decimal place before rounding to the level of precision you are looking for. In this case to round to 1 decimal place we would need to round to 2 decimal places and round to 1 decimal place on this new value.

Round(Round([Number], 0.01), 0.1)

This would result in the value you're looking for:

Regards,

Jonathan

Alteryx Partner

You can try this workaround : round([year 2 avg]*10,1)/10

Seems to work on my sample data

Meteoroid

I tried to apply this with a multi-field formula but it creates another anomaly (see highlighted cells)

Highlighted
Alteryx Partner

the jonathan's  solution may work better tan mine...

Bolide
Labels