Method of Gradient Descent in Python

Question

So I'm running a simple algorithm for gradient descent in python, trying to find the minimums of functions.

My code is reasonably straightforward, I feel it is intuitive. I've tried with the simple example of a second order polynomial.

def s(x):
    return x**2

def t(x):
    return 2*x

def steepd(a, alpha, epsilon):
    b = a - alpha*t(a)
    while abs(t(a)) > epsilon:
        if t(b) ==0:
            return b
        else: 
            a = b
            b = a - alpha*t(a)

         return b

   print(steepd(2,0.1,0.001))


x = np.arange(-10,10,0.01)
plt.plot(x,s(x))
plt.show()

I'm not getting anywhere near close to the minimum however, even by varying both alpha and epsilon significantly.

Can someone point out what the issue is?

Cheers.

Blender Blender · Accepted Answer · 2018-01-18T05:14:42

Your code terminates only if t(a) is within epsilon of zero, so there's no way it would produce a value that violates this constraint unless your indentation is broken (as it is in your question body) and your code really looks like this:

def steepd(a, alpha, epsilon):
    b = a - alpha*t(a)
    while abs(t(a)) > epsilon:
        if t(b) ==0:
            return b
        else: 
            a = b
            b = a - alpha*t(a)

        return b

Indentation in Python is important. Here, your while loop never runs more than once because of that return statement at the end and steepd is more or less just a - alpha*t(a). You'll need to properly indent the return statement for your code to function the way you want.

Method of Gradient Descent in Python

1 Answers