TensorFlow: Correct way of using steps in Stochastic Gradient Descent

Question

I am currently using TensorFlow tutorial's first_steps_with_tensor_flow.ipynb notebook to learn TF for implementing ML models. In the notebook, they have used Stochastic Gradient Descent (SGD) to optimize the loss function. Below is the snippet of the my_input_function:

def my_input_fn(features, targets, batch_size=1, shuffle=True, num_epochs=None):

Here, it can be seen that the batch_size is 1. The notebook uses a housing data set containing 17000 labeled examples for training. This means for SGD, I will be having 17000 batches.

LRmodel = linear_regressor.train(input_fn = lambda:my_input_fn(my_feature, targets), steps=100)

I have three questions -

1. Why is steps=100 in linear_regressor.train method above? Since we have 17000 batches and steps in ML means the count for evaluating one batch, in linear_regressor.train method steps = 17000 should be initialized, right?

2. Is number of batches equal to the number of steps/iterations in ML?

3. With my 17000 examples, if I keep my batch_size=100, steps=500, and num_epochs=5, what does this initialization mean and how does it correlate to 170 batches?

Answer 1

step is the literal meaning: means you refresh the parameters in your batch size; so for linear_regessor.train, it will train 100 times for this batch_size 1.
epoch means to refresh the whole data, which is 17,000 in your set.