Numeric Stable Implementation

Question

I need to calculate the normalized exponent of a vector in Matlab.

Just write

res = exp(V)/sum(exp(V)) 

the overflow in element V is greater than log (realmax) = 709.7827. (I'm not sure about the conditions of insufficient level.)

How do I implement it to avoid numerical instability?

Update: I got great answers on how to avoid overflow. However, I'm still glad to hear your thoughts on the possibility of code overflow.

Answer 1

The following approach avoids overflow by subtracting the exhibitors, and then using the exponent instead of dividing the exponents:

 res = 1./sum(exp(bsxfun(@minus, V(:), V(:).'))) 

As a rule, overflows can be avoided by working in the journal domain for as long as possible, and accepting the exponent only at the end.

Answer 2

The answer is pretty similar to your previous question. Use Math!

 exp(V)=exp(V-max(V))*exp(max(V)) sum(exp(V))=sum(exp(V-max(V))*exp(max(V)))=exp(max(V)*sum(exp(V-max(V)))) 

The combination of both:

 res=exp(V-max(V))*exp(max(V))/exp(max(V)*sum(exp(V-max(V)))=exp(V-max(V))/sum(exp(V-max(V))) 

Code that is resistant to input range:

 res=exp(V-max(V))/sum(exp(V-max(V)))