I searched for some of the Java Math source code sources. Especially tanh() , as I was curious to see how they implemented this one. However, what I found surprised me:
double tanh(double x) { ... if (ix < 0x40360000) { /* |x|<22 */ if (ix<0x3c800000) /* |x|<2**-55 */ return x*(one+x); /* tanh(small) = small */ ... }
As noted in the comment, the taylor tanh (x) series around 0 starts with:
tanh(x) = x - x^3/3 + ...
Then why does it look like they implemented it like:
tanh(x) = x * (1 + x) = x + x^2
This is clearly not a valid extension and even worse approximation than just using tanh(x) = x (which would be faster), as shown in this figure:

(The bold line is at the top. Another gray line is log(abs(x(1+x) - tanh(x))) . The sigmoid, of course, is tanh(x) .)
So, is it an implementation error, or is it a hack to fix some problem (for example, numerical problems that I really can't think of)? Note that I expect the result of both approaches to be exactly the same as if there werenโt enough mantissa bits to actually add 1 + x for x <2 ^. (- 55)
EDIT: I will include a link to the version of the code at the time of writing for future reference, as this may be fixed.
java c math floating-point ieee-754
Martijn courteaux
source share