How to initialise cost and iteration array within gradient descent class

Question

I'm trying to implement a class for a gradient descent function for multivariate linear regression. I want to plot the cost function against the number of iterations.

I'm trying to record each iteration value in an iteration array, and the value of the cost function for each iteration in a cost array. However, when I'm running the code, I get an error saying that my Gradient Descent class does not have a costArray or an iterationArray, even though I have already initialised both in my init class.

class GradientDescent():
    def __init__(self, learningRate=0.1, tolerance=0.02, iterations=500):
        self._learningRate = learningRate
        self._tolerance = tolerance
        self._iterations = iterations
        self._thetas = None
        self._iterationArray = []
        self._costArray = []    

    def fit(self, xs, ys):
        num_examples, num_features = np.shape(xs)
        self._thetas = np.ones(num_features)
        xs_transposed = xs.transpose()
        for i in range(self._iterations):
            self._iterationArray.append(i)
            diffs = np.dot(xs,self._thetas) - ys
            cost = np.sum(diffs**2) / (2*num_examples)
            self._costArray.append(cost)            
            gradient = np.dot(xs_transposed, diffs) / num_examples
            self._thetas = self._thetas-self._learningRate*gradient

            if cost < self._tolerance:
                return self._thetas

        return self._thetas

when I'm trying to plot the cost vs iterations graph by the following command using 'gd' as a Gradient Descent object:

plt.plot(gd.iterationArray,gd.costArray)

I'm getting the following error:

AttributeError: 'GradientDescent' object has no attribute 'iterationArray'

Is there something wrong with my initialisation? Or is there a simpler way I can record the iterations and the cost so that I can plot it?

thilakshiK thilakshiK · Accepted Answer · 2019-10-30T08:13:42

Try this plt.plot(gd._iterationArray,gd._costArray) . You have missed the underscore ( _ ) . It should be gd._iterationArray not gd.iterationArray.

You don't want a separate list to store your iterations because you already know it. So you can try something like this. plt.plot(list(range(iterations)), gd._costArray)

How to initialise cost and iteration array within gradient descent class

1 Answers