Logistic Regression, Gradient Descent Octave implementation

Question

I'm taking the Machine Learning class by Prof. Ng. There is a homework need to implement logistic regression gradient descent. And here is my code:

function [J, grad] = costFunction(theta, X, y)
%COSTFUNCTION Compute cost and gradient for logistic regression
%   J = COSTFUNCTION(theta, X, y) computes the cost of using theta as the
%   parameter for logistic regression and the gradient of the cost
%   w.r.t. to the parameters.

% Initialize some useful values
m = length(y); % number of training examples
[~,n] = size(X);
% You need to return the following variables correctly 
J = 0;
grad = zeros(size(theta));

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta.
%               You should set J to the cost.
%               Compute the partial derivatives and set grad to the partial
%               derivatives of the cost w.r.t. each parameter in theta
%
% Note: grad should have the same dimensions as theta
%

J = ((-y'*log(sigmoid(X*theta)))-((1-y)'*log(1-sigmoid(X*theta))))/m;
for j = 1:n
  temp_sum = 0;
  for i = 1:m
    temp_sum+=(sigmoid(X(i,:)*theta)-y(i))*X(i,j);
  endfor
  grad(j) = theta(j)-temp_sum;
endfor

% =============================================================

end

This is the formula that I'm trying to implement:

where h of x represent sigmoid function. I have check that sigmoid function is correct, but I still cant understand where is wrong in this algorithm. Please let me know if you find anything wrong.

Answer 1

I believe you were supposed to get the average of the gradient grad = grad / m as well, just like for the cost J. But it has been a while since I last did Andrew Ng's course, so I might be wrong.

Answer 2

Try this..This is what i did for my assignment. I think you were missing division by m.

function [J, grad] = costFunction(theta, X, y)
%COSTFUNCTION Compute cost and gradient for logistic regression
%J = COSTFUNCTION(theta, X, y) computes the cost of using theta as the
%parameter for logistic regression and the gradient of the cost
%w.r.t. to the parameters.

% Initialize some useful values
m = length(y); % number of training examples

% You need to return the following variables correctly 
J = 0;
grad = zeros(size(theta));

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta.
%               You should set J to the cost.
%               Compute the partial derivatives and set grad to the partial
%               derivatives of the cost w.r.t. each parameter in theta 
%
% Note: grad should have the same dimensions as theta
%
h=sigmoid(X*theta);
t1=(y'*log(h));
t2=(1-y)'*log(1-h);
J=-(t1+t2)/m;

for i =1:length(theta)
grad(i)=((h-y)'*X(:,i))/m;
endfor

% =============================================================

end