training step and validation step different in tensorflow

Question

I am using tensorflow callback for visualizing on tensorboard.

tensorboard = tf.keras.callbacks.TensorBoard(
    log_dir='logs',
    histogram_freq=1,
    write_graph=True,
    write_images=True,
    update_freq='epoch',
    profile_batch=2,
    embeddings_freq=1,
    )

I am using simple model and using data pipeline as input to the model. Here feature layer contains the feature_columns of each feature:

model = tf.keras.Sequential([
    feature_layer,
    tf.keras.layers.Dense(units = 12, activation='relu', use_bias = True, kernel_initializer= 'glorot_uniform', bias_initializer = 'zeros'),
    tf.keras.layers.Dense(units = 6, activation='relu', use_bias = True, kernel_initializer= 'glorot_uniform', bias_initializer = 'zeros'),
    tf.keras.layers.Dense(units = 2, activation='softmax')
    ])

I am using adam as optimizer, sparce categorical crossentropy as loss, and accuracy as metrics. Here is my tensorboard graph:

red line = train   
blue line = validation

my problem is why it is showing training step less then the validation step.

Hi @Aniket Bote, Can you show the epoch configuration when training? — TF_Support

Tensorflow Warrior Tensorflow Warrior · Accepted Answer · 2020-05-13T09:05:06

If you look into the tensorflow documentation for tf.keras.callbacks.TensorBoard, under arguments section for profile_batch, it is written as below -

profile_batch: Profile the batch(es) to sample compute characteristics. profile_batch must be a non-negative integer or a comma separated string of pair of positive integers. A pair of positive integers signify a range of batches to profile. By default, it will profile the second batch. Set profile_batch=0 to disable profiling. Must run in TensorFlow eager mode.

As you have set profile_batch=2, thus it showing results only for the second batch.

Example -

# profile a single batch, e.g. the 5th batch.
tensorboard_callback =
    tf.keras.callbacks.TensorBoard(log_dir='./logs', profile_batch=5)
model.fit(x_train, y_train, epochs=2, callbacks=[tensorboard_callback])
# run the tensorboard command to view the visualizations in profile plugin.

# profile a range of batches, e.g. from 10 to 20.
tensorboard_callback =
    tf.keras.callbacks.TensorBoard(log_dir='./logs', profile_batch='10,20')
model.fit(x_train, y_train, epochs=2, callbacks=[tensorboard_callback])
# run the tensorboard command to view the visualizations in profile plugin.

Set profile_batch=0 to disable profiling. Must run in TensorFlow eager mode.

training step and validation step different in tensorflow

1 Answers