How to write image data to tensorboard without opening new categories

Question

I wrote a Keras/Tensorflow callback that writes a confusion matrix to the Images tab in Tensorboard. This worked fine in TF 2.1. Unfortunately I had to convert it to TF 1.14, as other packages depend on this version.

Everything works fine (more or less), except for the Tensorboard Reports.

The Problem

As you can see in the screenshot below, there are many categories(? tags? channels? I am not sure about the terminology) instead of only one.

screenshot: multiple categories for images

Seemingly correlated to this issue, the training scalars like "val_loss" only plot the first datapoint and nothing afterwards. See second screenshot

screenshot: scalars showing one data point

Also, Tensorboard is printing the following error:
File <path/to/event-file> updated even though the current file is <path/to/new-event-file>

So I assume somehow my TF FileWriters are disagreeing on where to write.

Regarding the Confusion Matrix Callback: The writing function looks like this:

def _figure_to_summary(self, fig, step):

    # attach a new canvas if not exists
    if fig.canvas is None:
        matplotlib.backends.backend_agg.FigureCanvasAgg(fig)

    fig.canvas.draw()

    w, h = fig.canvas.get_width_height()

    img = np.fromstring(fig.canvas.tostring_rgb(), dtype=np.uint8, sep='')
    img = img.reshape((1, h, w, 3))

    with K.get_session().as_default():
        tensor = tf.constant(img)
        image = tf.summary.image(self.title, tensor).eval()
        self._summary_writer.add_summary(image, global_step=step)
        self._summary_writer.flush()

With fig being the matplot figure of the Confusion Matrix and step as an int, which should enable Tensorboard to add the little slider over the image to show the history of the matrix.

The model is trained as follows:

run_id = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
tb_path = os.path.join("tensor/", run_id)

tb_reporter = TensorBoard(tb_path)
summ_wr = FileWriter(tb_path)

conf_matr = ConfusionMatrix(va, TARGET_CLASSES.keys(), summ_wr, normalize=True)
cb_list = [tb_reporter, conf_matr]
model.fit(tr, epochs=500, validation_data=va, callbacks=cb_list)

Where summ_wr becomes self._summary_writer of the callback.

What I tried

I only tried changing the way the summary is written. Trying tf.merge_all(), opening, closing and reopening the FileWriter in various combinations, but nothing changed. When I deactivate the custom callback, the Tensorboard callback works as expected.

My Questions

How can I write Image Data into the same category every time?

Will the image get the scrollbar?

How can I fix the problem, that the Keras Tensorboard callback does not show its data?

I assume all the problems are related and solving one results in solving all of them, but I am completely stumped on how to do it.

I am thankful for any suggestions :)

Edit: I just found this question: Tensorboard Image Summaries This seems to solve the problem, unfortunately the answer does not have a lot of context, so I don't how to integrate the solution into the code.

RBock RBock · Accepted Answer · 2020-02-13T14:23:00

The solution I found consists of two parts.

First: I stole the FileWriter from the Tensorboard Callback from keras. Tensorboard.writer contains the FileWriter after set_model has been called.

Second: The Summary must be created via the constructor and be passed the tag-keyword. This forces the images into the same group and the writers do not interfere with each other.

I found the second part of the solution here: TensorBoard: How to write images to get a steps slider?

How to write image data to tensorboard without opening new categories

The Problem

What I tried

My Questions

1 Answers