I understand that a decimal can be so precise that it is stored as a float in the binary, but I donโt understand what happens in 7 decimal places in my 10/7 release.
In my first test release, I noticed that 5 (7th place) at the end of the float is v 4 (7th place), followed by 2 in double. I would think that the 7th place in the float will be 4, since the 8th place in the double is 2.
Float 1.4285715 Dual 1.4285714285714286
Then I performed the following test output using the float format up to 17 places. The 8th place in each is different - the float is 6, so the 7th place is rounded to 5, I suppose.
Float 1.428571462631225600 Dual 1.428571428571428600
In the third test, I tried live streaming with a string format to see what would happen. I got the same results as the second test.
If I have a simplified question about a complex floating point storage method - I apologize.
float f = 10/7f;
double d = 10/7d;
System.out.format
("Float %1s\nDouble %2s\n", f,d);
System.out.format
("Float %1$.17f\nDouble %2$.17f\n", f,d);
System.out.format
("Float %1s\nDouble %2s\n", (double)f,d);
source
share