The sum of positive and negative numbers does not equal zero.
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Hi all,
The sum of these two number isn't equal to 0. It equal to 4.07453626394272E-10, why was that happened?
- Labels:
- Dynamic Processing
- Macros
- Run Command
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Your data must have some underlying decimal places! You can use a Round function to get it to 0 - something like Round([External]+[SE26],0.01)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
That's how precise it gets. Your decimal places are quite precise - I would recommend to do your formula first, then at the end, use the Multi-Field tool to change the data type to fixed decimal, 19.0 to get it down if it helps.
If you round and do your formulas, you may end up with E.C.F. - error carried forward.
Alteryx ACE
https://www.linkedin.com/in/calvintangkw/
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
A double is a number stored as a fraction. On larger numbers (with decimals) it can be quite dififcult to get standard addition/subtraction to equal to 0. I'd recmmend swapping to fixed decimal 19.6.
