Geek Answers Handbook

SQL sets floating point precision

For an SQL int that converts to float, how to set the precision of a floating point number?

This is a selection that I would like to truncate to two or three decimal places:

AVG(Cast(e.employee_level as Float))avg_level,

Thanks!

+8
floating-point sql precision truncate
source share
2 answers

In TSQL, you can specify two different sizes for float , 24 or 53. This will set the accuracy to 7 or 15 digits, respectively.

If everything you want to do is truncated to a given number of decimal places, you can use ROUND , that is:

 ROUND(AVG(CAST(e.employee_level as float)), 3)
+22
source share

Generally, you cannot specify the number of digits after the decimal point for a floating point number. Floating-point data types store the closest floating-point approximation to any given value. The closest floating point approximation is unlikely to have the number of digits you want. Although you could suppress every digit after the third, it will change the appearance of the value, not the value itself.

Whole is a completely different story. An integer — stored, converted, or converted to a floating-point data type — will be stored exactly in a large range. For floating point data types, it is not necessary to store any fractional units for integers.

I would suggest the best for you

  • Avoid floating point integers unless you need fractional units or
  • distinguishes integers to decimal or numeric if you need fractional units or
  • handles display problems completely in the application code.
+1
source share

More articles:


All Articles