Here's a problem that over the past few hours has completely worried ...
I have an equation hardcoded in my program:
double s2; s2 = -(0*13)/84+6/42-0/84+24/12+(6*13)/42;
Each time I run the program, the computer spills out 3 as an answer, but doing the math manually, I get 4. Even further, after entering the equation in Matlab, I also get the answer 4. What happens here?
The only thing I can think of is what is going wrong is the rounding error. However, with a maximum of 5 rounding errors, combined with double precision math, my maximum error will be very small, so I doubt it is a problem.
Anyone who can offer any solutions?
Thanks in advance,
-Faken
source share