Ruby arithmetic

Question

Why does this code 7.30 - 7.20 in ruby ​​return 0.0999999999999996 , and not 0.10 ?

But if I write 7.30 - 7.16 , for example, everything will be fine, I will get 0.14 .

What is the problem and how can I solve it?

Answer 1

What Every Computer Scientist Should Know About Floating-Point Arithmetic

Answer 2

The problem is that some of the numbers, which we can easily write in decimal form, do not have an exact representation in a certain floating-point format implemented by the current equipment. A random way of saying this is that all integers, but not all fractions, because we usually store a fraction with an exponent of 2**e . So, you have 3 options:

• To complete. An unlimited result is always really very close, so the rounded result is invariably “perfect”. This is what Javascript does, and many people don’t even realize that JS does everything in a floating point.

• Use fixed point arithmetic. Ruby really makes this very easy; it is one of the only languages ​​that seamlessly moves to the Fixignum Bignum class as numbers get larger.

• Use a class designed to solve this problem, such as BigDecimal

In order to examine the problem in more detail, we can try to present your "7.3" in binary format. Part 7 is easy, 111, but how do we do it .3? 111.1 is 7.5, too large; 111.01 is 7.25, approaching. It turns out that 111.010011 is the "next nearest lower number", 7.296875, and when we try to fill in the missing .003125, in the end we find out that it is just 111.010011001100110011 ... forever, not represented in our chosen encoding in the final bit string.

Answer 3

The problem is that floating point is inaccurate . You can solve it using Rational, BigDecimal, or just simple integers (for example, if you want to store money, you can save the number of cents as an int instead of the number of dollars as a float).

BigDecimal can accurately store any number with a finite number of digits in the base of 10 and round numbers that do not have (so three-thirds are not integers).

Rational can accurately store any rational number and cannot store irrational numbers at all.

Answer 4

This is a common mistake in the way the numbers of floating point numbers are represented in memory.

Use BigDecimal if you need accurate results.

 result=BigDecimal.new("7.3")-BigDecimal("7.2") puts "%2.2f" % result 

Answer 5

It is interesting to note that a number that has several decimal places in one base can usually have a very large number of decimal places in another. For example, to calculate 1/3 (= 0.3333 ...) in base 10, an infinite number of decimal places is required, but only one decimal number in base 3. Likewise, many decimal places are required to express 1/10 (= 0.1) at the base of 2.

Answer 6

Since you are doing math with floating point, the return number is what your computer uses for precision.

If you need a closer answer, to the given accuracy, just a few float from this (for example, by 100), convert it to int, do the math and split.

There are other solutions, but I think this is the easiest, since rounding always seems a little worse to me.

This is set here, you may want to find some of the answers given earlier, for example: Working with precision problems in floating point numbers