Strange behavior when having fixed decimal in the expression

I thiunk this is linked with the type of data you use, I see it as text (V_Wstring), try putting it in fixed decimal or double, it should work, I think in text it lokks for the exact string, meaning without a 0 at the end it won't work!

tell me if it helped, or if it did not, send a dataset so taht I can look into it!

Ladarthure

hi again,

have checked your workflow, it seems a bit odd, I do have similar behaviors on my version (2019.1), I tried a few things with random numbers and it seems to work fine, but I have to say I don't fully understand why it's not working as intended in your case.

Decimal Places Issue - Samuel.yxmd

ChrisK

@Samuel_ToWe have changed the way we converted text to numbers in our formula processor. It is still a bad idea to compare floating point numbers for exact equality, because the way they are computed can lead to almost invisible differences. Even so, it was annoying that 0.6 didn't come out to be the same number as 0.600. We fixed this in version 2022.1. Can you give it a try and see what you get?

The way this can happen is that with "0.6", somewhere in the conversion it computes 6/10 to get the value you want. With "0.600" it does the computation 600/1000. It happens that those two computations didn't come out to be exactly the same. It could be that it really computed 600 / 10 / 10 / 10, for example. They way the computation is done makes a critical difference.

The code change we made in 22.1 uses the same conversion from text to numbers in the formula library as is used elsewhere in Alteryx code. This increases your odds that numbers that look the same will actually be the same. However, it is never safe to trust that two different paths that should give the same number really will give exactly the same number.

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703