Get Inspire insights from former attendees in our AMA discussion thread on Inspire Buzz. ACEs and other community members are on call all week to answer!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

How does the Formula tool handle large numbers in the INT64 data type?

Claje
14 - Magnetar

I'm hoping to get a detailed answer to this question.

I stumbled across this post today:
https://community.alteryx.com/t5/Data-Preparation-Blending/Data-cleansing-tool-giving-warnings-on-fi...

 

Out of curiosity, I tried replicating what happens if you have a field right near the upper limit of the INT64 field type, and run it through a formula tool.


What I observed is that I got the conversion error message:  Formula (10) Field1: -9223372036854775808 does not fit in the type Double.


I also found that if I tried to do a calculation like [Field1]*-1, the output did not change.  However, if I did something like [Field1]/1000, I still get a conversion error, but the output is processed.

 

Is Alteryx converting numbers to Double inside the Formula tool? Why? Is there any way to suppress this message that only occurs due to this internal process?

 

 

EDIT: I also just tried using a Formula tool to create a new value in an INT64 field that is near the supposed max length

9223372036854775806


The observed error message is particularly confusing, and the final output is NULL

 

Formula (2) INT64Field: 9223372036854777856.000000 does not fit in the type Int64

 

 

Notably, it looks like the number in the conversion error is much smaller than the supposed value, and that it rounded in a particularly odd way

2 REPLIES 2
jrgo
14 - Magnetar

@Claje,

 

Yes; under the hood, Alteryx processes all numerical fields as doubles. Here's a couple ways to suppress though.

 

image.png

harsh_alang
Alteryx
Alteryx

Starting version 23.1 Alteryx has added INT64 support in Formulas. Users will now be able to leverage large numbers in their calculations without losing precision.

HarshA
Labels