Nelder-Mead optimization with equality constraints

Question

I would like to use the function fminsearch of matlab to search for the best hyperparameters of my SVM with a weighted RBF kernel classifier. fminsearch uses the Nelder-Mead simplex method.

Let's say I have the following hyperparameters: C, gamma, w₁....w_n where w_i are the weights of the kernel.

Additionally, I have the constraint that sum(w_i) = 1, i.e. all weights must sum up to one.

Is there a possibility to use Nelder-Mead with this equality constraint? I know that there is the fminsearchbnd method for Matlab but I think it can handle only boundary inequality constraints.

Edit: I'm using a SVM classifier and the weights are used in a weighted RBF kernel (one weight for each feature). The parameters to estimate are thus C, gamma and the weights. The cost function is the accuracy.

Answer 1

Can you substitute out one of the w(i)? That means, replace e.g. w1 by 1-w2-w3-... (and drop the constraint). Otherwise have a look at fmincon which allows explicit constraints. In addition you may need 0 <= w(i) <= 1.