I am trying to optimize a certain function using the Nelder-Mead method and I need help understanding some of the arguments. I am fairly new to the world of numerical optimizations so, please, forgive my ignorance of what might be obvious to more experienced users. I note that I already looked at minimize(method=’Nelder-Mead’) and at scipy.optimize.minimize but it was not of as much help as I would have hoped. I am trying to optimize function $f$ under two conditions: (i) I want the optimization to stop once the $f$ value is below a certain value and (ii) once the argument is around the optimal value, I don't want the optimizer to increase the step again (i.e., once it gets below the threshold value and stays below for a couple of iterations, I would like the optimization to terminate). Here is the optimization code I use:
scipy.optimize.minimize(fun=f, x0=init_pos, method="nelder-mead",
options={"initial_simplex": simplex,
"disp": True, "maxiter" : 25,
"fatol": 0.50, "adaptive": True})
where f is my function (f : RxR -> [0,sqrt(2))). I understand that x0=init_pos are initial values for f, "initial_simplex": simplex is the initial triangle (in my 2D case), "maxiter" : 25 means that the optimizer will run up to 25 iterations before terminating.
Here are things I do not understand/I am not sure about:
The website 1 says "fatol: Absolute error in func(xopt) between iterations that is acceptable for convergence." Since the optimal value for my function is f(xopt)=0, does
"fatol": 0.50mean that the optimization will terminate once the f(x) will have the value of 0.5 or less? If not, how do I modify the condition for termination (in my case, how do I assure that it does stop once f(x)<=0.5)? I am ok if the optimizer runs a few more iterations around the region giving <0.5 but right now it tends to jump out of the near optimal region in a completely random way and I would like to be able to prevent it (if possible).Likewise, as far as I understand, "xatol: Absolute error in xopt between iterations that is acceptable for convergence." means that the optimization will terminate once the difference between the optimal and the present arguments is at most
xatol. Since in principle I do not know a-priori what the xopt is, does it mean in practice that once |x_n - x_(n+1)|, the optimizer will stop? If no, is there a way of adding a constraint to stop the function once it is near the optimal point?
I will appreciate if someone can either answer or give me a better reference than the SciPy documentation.
