Taking a log of very small values ​​using numpy / scipy in Python

Question

I have an Nx1 array that corresponds to a probability distribution, i.e. the sum of the elements adds up to 1. This is represented as a normal numpy array. Since N can be relatively large, such as 10 or 20, many of the individual elements are pretty close to 0. I find that when I take log (my_array) I get the error "FloatingPointError: invalid value found in the log" . Note that this is after setting seterr (invalid = 'raise') to numpy intentionally.

How can I solve this digital problem? I would like to present the vectors corresponding to the probability distribution and their registration log without rounding to 0, since then I end up taking log (0), which causes an error.

thank.

Answer 1

Which is pretty close to zero?

>>> np.log(0)
-inf
>>> 0.*np.log(0)
nan
>>> np.log(1e-200)
-460.51701859880916
>>> 1e-200*np.log(1e-200)
-4.6051701859880914e-198

One solution is to add a small positive number to all the probabilities in order to limit them far enough from zero.

The second solution is to explicitly process zeros, for example, replace 0. * np.log (0) with zeros in the resulting array or include only those points that have a nonzero probability in the probability array

Answer 2

You can simply leave the tails in accordance with the required accuracy.

eps = 1e-50
array[array<eps]=eps
log(array)

Answer 3

" " 0, ? Python , 10 ^ - :

>>> log(0.0000000000000000000000000001)
-64.472382603833282

, ? , ?

Answer 4

Depending on what you do later, you can use another transformation that did not explode at zero values, such as a log. Perhaps a sigmoid function or something else with a well-defined Jacobian.

If you just want to visualize the data, you can always add a small value before taking the log.