cancel
Showing results for 
Search instead for 
Did you mean: 

Alteryx designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.

Floating point clarification

Highlighted
Meteoroid

After reading up on floating point errors, I'm still unable to understand how it's creating the behavior we're seeing in some of our workflows when the same number behaves differently across different records.

 

For example, we have a double field and two records both have the number 5 in that field.  They both pass through a formula that codes them into buckets where they are essentially flagged as >=5 or <5.  I understand that this doesn't work perfectly because of floating point rounding, but shouldn't it always work imperfectly in the same way for the same number?  In other words, if two of the same number (5) are both passing through the same formula, shouldn't they have the same outcome (regardless of whether the outcome is right or wrong)?

 

Similar issue when we group by a double and sometimes the same number would show up in different groups.

 

Am I thinking about floating point incorrectly?

 

 

Alteryx Certified Partner

Could you post the workflow with at least a sample of the data that is producing this inconsistency? Options>Export Workflow will save a .yxzp that included the data input.

Meteoroid

Hi Charlie - the data is from a database so I can't upload it.  When I copy the data into Excel everything ends up correcting itself.  I've attached some screenshots of what I'm seeing if that's helpful.  

Alteryx Certified Partner

Are you able to add an Auto Field or Select tool to force the field types in this workflow? Otherwise, in your query, you could cast the fields to specific field types.

Meteoroid

Yes, we're using double but could use a fixed decimal if necessary.  I was more curious about why it only happens on certain records - my understanding of floating point is that the number gets stored to as much precision as the field type would allow, but that would mean that every instance of 5 would round the same.  Is that incorrect?

Labels