Scipy.optimize.minimize = 'SLSQP' method ignores the restriction

Question

I use SciPy for optimization, and the SLSQP method seems to ignore my limitations.

In particular, I want x [3] and x [4] to be in the range [0-1]

I get the message: "Uneven restrictions are incompatible"

Here are the execution results, followed by a code example (uses a dummy function):

status: 4 success: False njev: 2 nfev: 24 fun: 0.11923608071680103 x: array([-10993.4278558 , -19570.77080806, -23495.15914299, -26531.4862831 , 4679.97660534]) message: 'Inequality constraints incompatible' jac: array([ 12548372.4766904 , 12967696.88362279, 39928956.72239509, -9224613.99092537, 3954696.30747453, 0. ]) nit: 2 

Here is my code:

 from random import random from scipy.optimize import minimize def func(x): """ dummy function to optimize """ print 'x'+str(x) return random() my_constraints = ({'type':'ineq', 'fun':lambda(x):1-x[3]-x[4]}, {'type':'ineq', 'fun':lambda(x):x[3]}, {'type':'ineq', 'fun':lambda(x):x[4]}, {'type':'ineq', 'fun':lambda(x):1-x[4]}, {'type':'ineq', 'fun':lambda(x):1-x[3]}) minimize(func, [57.9499 ,-18.2736,1.1664,0.0000,0.0765], method='SLSQP',constraints=my_constraints) 

EDIT - The problem persists when even when you delete the first constraint.

The problem persists when I try to use border variables. i.e.,

 bounds_pairs = [(None,None),(None,None),(None,None),(0,1),(0,1)] minimize(f,initial_guess,method=method_name,bounds=bounds_pairs,constraints=non_negative_prob) 

Answer 1

I know this is a very old question, but I was intrigued.

When does this happen?

This problem occurs when the optimization function is not reliably differentiable. If you use a nice smooth function like:

 opt = numpy.array([2, 2, 2, 2, 2]) def func(x): return sum((x - opt)**2) 

The problem goes away.

How to impose strict restrictions?

Please note that none of the limited algorithms in scipy.minimize guarantees that a function will never be evaluated outside of the restrictions. If you need it, you should use conversions. So, for example, to ensure that no negative values ​​for x [3] are ever used, you can use the transformation x3_real = 10^x[3] . Thus, x [3] can be any value, but the variable you use will never be negative.

Deeper analysis

Examining the Fortran code for slsqp provides the following information about when this error occurs. The procedure returns the MODE variable, which can take these values:

 C* MODE = -1: GRADIENT EVALUATION, (G&A) * C* 0: ON ENTRY: INITIALIZATION, (F,G,C&A) * C* ON EXIT : REQUIRED ACCURACY FOR SOLUTION OBTAINED * C* 1: FUNCTION EVALUATION, (F&C) * C* * C* FAILURE MODES: * C* 2: NUMBER OF EQUALITY CONTRAINTS LARGER THAN N * C* 3: MORE THAN 3*N ITERATIONS IN LSQ SUBPROBLEM * C* 4: INEQUALITY CONSTRAINTS INCOMPATIBLE * C* 5: SINGULAR MATRIX E IN LSQ SUBPROBLEM * C* 6: SINGULAR MATRIX C IN LSQ SUBPROBLEM * 

The part that assigns mode 4 (which is the error you get) is as follows:

 C SEARCH DIRECTION AS SOLUTION OF QP - SUBPROBLEM CALL dcopy_(n, xl, 1, u, 1) CALL dcopy_(n, xu, 1, v, 1) CALL daxpy_sl(n, -one, x, 1, u, 1) CALL daxpy_sl(n, -one, x, 1, v, 1) h4 = one CALL lsq (m, meq, n , n3, la, l, g, a, c, u, v, s, r, w, iw, mode) C AUGMENTED PROBLEM FOR INCONSISTENT LINEARIZATION IF (mode.EQ.6) THEN IF (n.EQ.meq) THEN mode = 4 ENDIF ENDIF 

So basically you can see that he is trying to find the direction of the descent, if the restrictions are active, he is trying to get a derivative estimate for the restriction and fails with the singular matrix in the subproblem lsq ( mode = 6 ), then this explains that if all the equations the restrictions were evaluated, and none of them gave successful descent directions, this should be a contradictory set of restrictions ( mode = 4 ).