1x1 convolution as classification layer in Pytorch

Question

I am trying to classify image patches into 10 different categories using a neural network. My idea (borrowed from this article is to use the first 5 layers of a pretrained VGG network and apply a 1x1 convolution to this encoder. So, given the first 5 layers of a VGG network:

Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU(inplace=True)
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU(inplace=True)
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (6): ReLU(inplace=True)
    (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): ReLU(inplace=True)
    (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (13): ReLU(inplace=True)
    (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): ReLU(inplace=True)
    (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): ReLU(inplace=True)
    (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (20): ReLU(inplace=True)
    (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (22): ReLU(inplace=True)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): ReLU(inplace=True)
    (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (27): ReLU(inplace=True)
    (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (29): ReLU(inplace=True)
    (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)

How can I add a layer to classify each activation into n_classes? I am trying with a 1x1 convolution from 512 (number of filters in last layer) to n_classes, something like this:

nn.Sequential(
    nn.Conv2d(in_channels=512, out_channels=n_classes, kernel_size=1),
    nn.Softmax(dim=1)
    )

This network gives (batch_size, n_classes H // 32, W // 32) outputs. However, all outputs for each class are equal for a given image, I mean that, for a given batch, B and class C, all values are the same.

I have tried training with CrossEntropyLoss and even converting to logits and applying BCELoss, but without success. I also have tried other networks in my train loop and this do not happens.

Any clue?

Thanks

Ricard Borràs Ricard Borràs · Accepted Answer · 2019-10-12T19:06:54

0

votes

Finally solved to manage myself. It was a initialitzation of last layer weights

Best

1x1 convolution as classification layer in Pytorch

1 Answers