What does learning algorithm output in linear regression?

0
votes

Reading course notes of Andrew NG's machine learning course it states for linear regression :

Take a training set and pass it into a learning algorithm. The algorithm outputs a function h (the hypothesis). h takes an input and tries to output estimated value y.

It then goes on to say :

present h as : h theta(x) = theta0 + theta1x

Does this not mean the hyptohesis was not outputted by the learning algorithm, instead we just defined it as h theta(x) = theta0 + theta1x

Instead of "Take a training set and pass it into a learning algorithm. The algorithm outputs a function h (the hypothesis)." should the statement be "Take a training set and pass it into a learning algorithm. The algorithm outputs value(s) which make the hypothesis as accurate as possible" ?

2
machine-learninglinear-regressiongradient-descent
Does this not mean the hyptohesis was not outputted by the learning algorithm, instead we just defined it as h theta(x) = theta0 + theta1x - what do you mean here? theta(x) = theta0 + theta1x is your hypothesis. - cel
@cel the hypothesis is defined as "h theta(x) = theta0 + theta1x" . The learning algorithm did output this hypothesis as stated in course notes. - blue-sky
I don't get the difference between the two statements. Returning a function or returning all necessary parameters for a parametric function is basically the same information. - cel
@cel I meant to say in previous comment "@cel the hypothesis is defined as "h theta(x) = theta0 + theta1x" . The learning algorithm did not output this hypothesis as stated in course notes. " I distinguish between returning a function and returning the evaluation of a function. The learning algorithm appears to return value, not a function. - blue-sky
I think cel is correct here, but we cannot see your system outputs. It is highly likely that you are getting the coefficients of your hypothesis as simply two raw numbers. I assume the task is for you to take the coefficients and build the equation. - roganjosh

2 Answers

1
votes

In principle you are right here. A true learning algorithm as defined in learning theory is an algorithm that gets labelled instances and a whole class of possible hypotheses as input and then chooses one hypothesis as an output.

So strictly speaking, an algorithm that outputs the predictions is not a learning algorithm. But of course such an algorithm can be split into a learning algorithm - the algorithm that actually learns the parameters, here the thetas. and a prediction algorithm that transforms some input instances to our predictions which are then returned to the caller.

1
votes

For the case of linear regression you want your learning algorithm to output a linear fucnction.

That is h(x) = theta0 + theta1x.

In this case the learning algorithm learns the optimal theta0 and theta1 to fit your training data.

If you wanted your learning algorithm to learn a 3rd degree polynomial the output of your learning model would be a, b, c and d such that

h(x) = ax3 + bx2 + cx + d

But your assertion is correct, the learning algorithm chooses the best parameters to minimize the cost of an error function. Usually this is squared error + some regularization factors.