It is not uncommon in programming to have a situation where what is most convenient for your function and what some library call expects of it don't agree. The normal resolution to such a problem is to write a small layer in between that allows the two to talk; an interface.
From help ga:
X = GA(FITNESSFCN,NVARS) finds a local unconstrained minimum X to the
FITNESSFCN using GA. [...] FITNESSFCN accepts a vector X of size
1-by-NVARS, and returns a scalar evaluated at X.
So, ga expects vector input, scalar output, whereas you have a structure going in and out. You would have to write the following (sub)function:
function Y = wrapper_Objfun(X, in)
in.a = X; %# variable being optimized
out = YOUR_REAL_FUNCTION(in); %# call to your actual function
Y = out.b(2,3); %# objective value
end
and then the call to ga will look like
X = ga(@(x) wrapper_Objfun(x,in), N);
where N is however large in.a should be.
Also have a read about it in Matlab's own documentation on the subject.