L-BFGS-B does not satisfy given constraint

Question

I try to find optimized weight values of my model by using minimize function in scipy. As seen below the code, I define my error function returning one minus f1 score of the model.

 def err_func(weights,x,y):
        undetected=0
        correct=0
        incorrect=0
        results=fun(weights,x)
        for i in range(0,len(results)):
            if(results[i]==y[i]):
                correct+=1
            elif(not (results[i]==y[i])):
                incorrect+=1
        undetected=len(y)-(correct+incorrect)
        precision=float(correct) / float(correct + incorrect)
        recall=float(correct) / float(correct + incorrect + undetected)
        f1=2 * precision * recall / (precision + recall)
        return 1.0-f1

I use constraints that each value in weights between zero and one, and sum of weights are equals to one. These definitions are as below:

cons = ({'type': 'eq', 'fun': lambda x: 1 - sum(x)})
bnds = tuple((0.0, 1.0) for x in weights)
eps=1e-2

But while running minimze method, my function does not satisfy the constraint.

from scipy.optimize import minimize
res = minimize(err_func, weights,method='L-BFGS-B', args=(x,y),constraints=cons,bounds=bnds,options = {'eps':eps,'maxiter':100}) 
print res
test_weights=res.x
print sum(test_weights)

I got such an output, sum of weights are larger than one. What am I missing?

>    fun: 0.4955555555555555  hess_inv: <11x11 LbfgsInvHessProduct with
> dtype=float64>
>       jac: array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.])   message: 'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL'
>      nfev: 24
>       nit: 1    status: 0   success: True
>         x: array([ 0.        ,  0.22222222,  0.        ,  1.        ,  1.        ,
>         0.11111111,  1.        ,  1.        ,  1.        ,  0.        ,  1.        ])
> 6.33333333333

Erwin Kalvelagen Erwin Kalvelagen · Accepted Answer · 2017-08-28T13:36:38

L-BFGS-B only supports bound constraints (that is what the second 'B' means). General constraints are not supported by this method.

Excerpt from scipy docs:

Parameters:
    ...
    constraints : dict or sequence of dict, optional
        ...
        Constraints definition (only for COBYLA and SLSQP)

L-BFGS-B does not satisfy given constraint

1 Answers