python - Tensorboard Image Summaries

Question

I use Matplotlib to create custom t-SNE embedding plots at each epoch during trainging. I would like the plots to be displayed on Tensorboard in a slider format, like this MNST example:

But instead each batch of plots is displayed as separate summaries per epoch, which is really hard to review later. See below:

It appears to be creating multiple image summaries with the same name, so appending _X suffix instead of overwriting or adding to slider like I want. Similarly, when I use the family param, the images are grouped differently but still append _X to the summary name scope.

This is my code to create custom plots and add to tf.summary.image using custom plots and add evaluated summary to summary writer.

def _visualise_embedding(step, summary_writer, features, silhouettes, sample_size=1000):
    '''
    Visualise features embedding image by adding plot to summary writer to track on Tensorboard
    '''
    # Select random sample
    feats_to_sils = list(zip(features, silhouettes))
    shuffle(feats_to_sils)
    feats, sils = zip(*feats_to_sils)
    feats = feats[:sample_size]
    sils = sils[:sample_size]

    # Embed feats to 2 dim space
    embedded_feats = perform_tsne(2, feats)

    # Plot features embedding
    im_bytes = plot_embedding(embedded_feats, sils)

    # Convert PNG buffer to TF image
    image = tf.image.decode_png(im_bytes, channels=4)

    # Add the batch dimension
    image = tf.expand_dims(image, 0)
    summary_op = tf.summary.image("model_projections", image, max_outputs=1, family='family_name')
    # Summary has to be evaluated (converted into a string) before adding to the writer
    summary_writer.add_summary(summary_op.eval(), step)

I understand I might get the slider plots I want if I add the visualise method as an operation to the graph so as to avoid the name duplication issue. But I need to be able to loop through my evaluated tensor values to perform t-SNE to create the embeddings...

I've been stuck on this for a while so any advise is appreciated!

Soph Soph · Accepted Answer · 2018-10-18T08:32:02

This can be achieved by using tf.Summary.Image()

For example:

    im_summary = tf.Summary.Image(encoded_image_string=im_bytes)
    im_summary_value = [tf.Summary.Value(tag=self.confusion_matrix_tensor_name, 
    image=im_summary)]

This is a summary.proto method so it was obvious to me at first as the method definition is not accessible through Tensorflow. I only realised its functionality when I found a code snippet of it being used on github.

Either way, it exposes image summaries as slides on Tensorboard like I wanted. 💪

python - Tensorboard Image Summaries

1 Answers