noise in image segmentation result

Question

I am doing a image segmentation task based on deep convolutional neural network. The network structure is from this paper, and the structure can be seen in the picture:FCN used in image segmentation. The network is designed based on Fully Convolutional Network and DCAN.

The data set is the public benchmark dataset of Gland Segmentation Chalenge Contest in MICCAI 2015(also named as Warwick-QU dataset.)

I use this network to do the segmentation task. However in my result, I always noticed a fixed pattern of noise: the little white crossing in the predicted image:Prediction results with crossing shape noise

Can someone please explain what does these noise pixels mean? Are they the common noises in image processing? It is worth noting that this phenomena does not appear only in a specific image, but for the whole prediction.

That has also happened to me before, but I've never found a satisfactory explanation. Can I ask what cost function you are using? — Ash
Thanks for the reply. I am using categorical_crossentropy. Are you also doing image segmentation on MICCAI 2015 dataset? — Wentai Chen
Hello Ash, I found the trick. These crossing-shape noises are caused by the deconvolutional layer in my network. Let's say we have a feature map of 32x32 and we want to get the feature map with the size of 256x256, so we have to use the deconvolutional layer with a stride of 8x8 to do the deconvolution, thus a lot of zero pixels are added to the feature map, that is why these crossing shape occurs. To deal with this, we have to enlarge our kernel size twice as the stride. In the above example we need choose a 16x16 kernel so that our filter will not fall into those zeros. — Wentai Chen
FYI, the operation of deconvolution is illustrated here: deconvolution — Wentai Chen
Thanks for this notification Ash, I already updated my answer. — Wentai Chen

Wentai Chen Wentai Chen · Accepted Answer · 2018-04-05T14:19:56

These crossing-shape noises are caused by the deconvolutional layer in my network, and it is solved by changing the kernel size in deconvolutional layer.

First we should understand how does the deconvolutonal layer work. A clear illustration can be found here. What we need to bear in mind is that the stride in deconvolution helps to enlarge the size of the feature map.

Back to this example. Let's say we have a feature map of 32x32 and we want to get the feature map with the size of 256x256, so we have to use the deconvolutional layer with a stride of 8x8 to do the deconvolution, thus a lot of zero pixels are added to the feature map, that is why these crossing shape occurs. To deal with this, we have to enlarge our kernel size twice as the stride. In the above example we need choose a 16x16 kernel so that our filter will not fall into those zeros.

The result image can be shown here: result
We can see clearly that the crossing-shape noises are removed compared with the picture attached in the question.

noise in image segmentation result

2 Answers