What is the R equivalent of Matlab's fminunc function?

Question

In order to compute the optimal theta e.g. in logistic regression, I have to create a costFunction (the function to be minimized) which is then passed to fminunc in order to obtain the optimal theta. Also, if the gradient of costFunction can be computed, I set the 'GradObj' option to 'on' using

options = optimset('GradObj','on');

and code the costFunction so that it returns, as a second output argument, the gradient value g of X. Then I give

[theta, cost] = fminunc(@(t)(costFunction(t, X, y)), initial_theta, options);

where X is the data matrix and y the response. How can I implement the above in R?

Answer 1

Take a look at the optim function. It can do unconstrained minimization using method = 'L-BFGS-B' and you can specify an analytical function to compute the gradient as well

EDIT. As Ben has pointed out correctly, fminunc does unconstrained optimization, which can also be achieved using the optim function choosing Nelder-Mead or BFGS. Moreover, I also noticed from the documentation of fminunc that it does large-scale optimization using trust region methods. There is an R package trust that I believe does the same thing. I would recommend taking a look at the optimization task view of R.

Answer 2

In the R, you can use the function nlminb in the R to do constrained optimization!

nlminb(start, objective, gradient = NULL, hessian = NULL, ..., scale = 1, control = list(), lower = -Inf, upper = Inf)

The start is a vector include all initial value of parameters. objective is the cost function or any other function that you want to minimize.

Answer 3

Have you tried the ucminf function of the ucminf package? If yours is an unconstrained problem (and in many cases you can easily convert constrained problems into unconstrained ones), the ucminf is quite similar to Matlab's fmincon. The two are similar in the sense that both use a trust-region type monitoring. Unlike fmincon (that relies on the interior-point algorithm), ucminf is based on a quasi-Newton type of algorithm. However, usminf provides you with the same types of controls as fmincon. From my experience, ucminf is pretty good at replicating fmincon's output, give it a shot.