ST_Area calculations seem wrong

[rferriman]

I am getting very different area values from the ST_Area function depending on how many decimal places are reflected in the the lat long values used to create the polygon.  

For example I have a polygon with only 4 points and is roughly shaped like a rectangle.  If I change the points to 7 decimal places of precision I get an area of 16.944 square meters.  If I have 8 decimal places it is 15.33 and with 9 it is 17.23.  These changes are far to big for the small changes in precision.  Also, changing from 14 to 15 decimal places decreases the area 2%.  

Is there something I am overlooking or could these values actually be correct?

The points are-

Sequence  Lat                               Lon

0                40.1207637051088     -82.9919984315984

1                40.1207333439761     -82.9919988998749

2                40.1207338792313     -82.9920567790322

3                40.1207642420278     -82.9920562140683

The area calculations were-

Decimal Places   Area (square meters)

6                           16.505

7                           16.944

8                           15.332

9                           17.232

10                         16.593

11                         16.418

12                         15.518

13                         16.181

14                         15.745

15                         15.433

These runs were all performed in Designer version 2022.3.1.508

The same results come back with AMP on and off

[CharlieS]

My hypothesis is that there are some floating-point operations deep within the area calculations that Alteryx performs that will happen no matter the tools or methods we use within Alteryx.

The GIS answer for this situation is:

1) Decimal places to this length are superfluous. There's good reading on the subject here:
coordinates - Measuring accuracy of latitude and longitude - Geographic Information Systems Stack Ex...

2) Polygons of this size should probably use local surveyors for accurate measurements, on a different projection (like a local UTM), and be calculated with a specialized GIS platform (like ArcGIS Pro). 

Long story short: stick to 6 decimal places as that's the limit of precision we are working with in this platform and calculation methods.

[Hollingsworth]

See also https://xkcd.com/2170

As Charlie said, storing latitude and longitude as Fixed Decimal with 6 digits of precision is the way.