Plot several graphs in Pytorch tensorboard

3

votes

I am training a dynamic neural network, meaning that each epoch I tweak the architecture and get a different computational graph. I want to plot the graph for each epoch using tensorboard, but when I use SummaryWriter.add_graph() at the end of each epoch it simply overwrites the previous one.

Any ideas how to plot several graphs using pytorch + tensorboard? It seems achievable as each graph has a “tag” but I found no option to change this tag to plot several of them.

Thanks, Elad

pytorchtensorboard

0

votes

If you still want to use SummaryWriter, there is the optionn of using the method "add_scalars":

Example:

summary.add_scalars(f'loss/check_info', {
    'score': score[iteration],
    'score_nf': score_nf[iteration],
}, iteration)

0

votes

Instead of using the "tag" feature, you can use the "run" feature. To do so, you have to open tensorboard from a directory within which you stored your summaries in distinct sub-directories.

In your example, you could save the summary of the first epoch at the directory "tensorboard_log_dir/epoch_1", and then save the summary of the second epoch at the directory "tensorboard_log_dir/epoch_2", etc.

That way, when using tensorboard --logdir=tensorboard_log_dir, you'll be able to switch from one computational graph to another via the "run" widget.

Here's a reproducible example:

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.tensorboard import SummaryWriter

dummy_input = (torch.zeros(1, 3),)

# Two different architectures (PyTorch)
class oneLinear(nn.Module):
    def __init__(self):
        super(oneLinear, self).__init__()
        self.l1 = nn.Linear(3, 5)

    def forward(self, x):
        x = self.l1(x)
        return x

class twoLinear(nn.Module):
    def __init__(self):
        super(twoLinear, self).__init__()
        self.l1 = nn.Linear(3, 5)
        self.l2 = nn.Linear(5, 5)

    def forward(self, x):
        x = self.l1(x)
        x = F.relu(self.l2(x))
        return x

# add graph into 2 distinct subdirectories
with SummaryWriter('./tensorboard_log_dir/oneLinear') as w:
    w.add_graph(oneLinear(), dummy_input)

with SummaryWriter('./tensorboard_log_dir/twoLinear') as w:
    w.add_graph(twoLinear(), dummy_input)

Plot several graphs in Pytorch tensorboard

2 Answers