Training neural network for image segmentation

Question

I have one set of original image patches (101x101 matrices) and another corresponding set of image patches (same size 101x101) in binary which are the 'answer' for training the neural network. I wanted to train my neural network so that it can learn, recognize the shape that it's trained from the given image, and produce the image (in same matrix form 150x10201 maybe?) at the output matrix (as a result of segmentation).

Original image is on the left and the desired output is on the right.

enter image description here

So, as for pre-processing stage of the data, I reshaped the original image patches into vector matrices of 1x10201 for each image patch. Combining 150 of them i get a 150x10201 matrix as my input, and another 150x10201 matrix from the binary image patches. Then I provide these input data into the deep learning network. I used Deep Belief Network in this case.

My Matlab code for setup and train DBN as below:

%train a 4 layers 100 hidden unit DBN and use its weights to initialize a NN

rand('state',0)

%train dbn

dbn.sizes = [100 100 100 100];
opts.numepochs =   5;
opts.batchsize =   10;
opts.momentum  =   0; 
opts.alpha     =   1;
dbn = dbnsetup(dbn, train_x, opts);
dbn = dbntrain(dbn, train_x, opts);

%unfold dbn to nn

nn = dbnunfoldtonn(dbn, 10201);
nn.activation_function = 'sigm';

%train nn

opts.numepochs =  1;
opts.batchsize = 10;

assert(isfloat(train_x), 'train_x must be a float');
assert(nargin == 4 || nargin == 6,'number ofinput arguments must be 4 or 6')

loss.train.e               = [];
loss.train.e_frac          = [];
loss.val.e                 = [];
loss.val.e_frac            = [];
opts.validation = 0;
if nargin == 6
    opts.validation = 1;
end

fhandle = [];
if isfield(opts,'plot') && opts.plot == 1
    fhandle = figure();
end

m = size(train_x, 1);

batchsize = opts.batchsize;
numepochs = opts.numepochs;

numbatches = m / batchsize;

assert(rem(numbatches, 1) == 0, 'numbatches must be a integer');

L = zeros(numepochs*numbatches,1);
n = 1;
for i = 1 : numepochs
tic;

kk = randperm(m);
for l = 1 : numbatches
    batch_x = train_x(kk((l - 1) * batchsize + 1 : l * batchsize), :);

    %Add noise to input (for use in denoising autoencoder)
    if(nn.inputZeroMaskedFraction ~= 0)
        batch_x = batch_x.*(rand(size(batch_x))>nn.inputZeroMaskedFraction);
    end

    batch_y = train_y(kk((l - 1) * batchsize + 1 : l * batchsize), :);

    nn = nnff(nn, batch_x, batch_y);
    nn = nnbp(nn);
    nn = nnapplygrads(nn);

    L(n) = nn.L;

    n = n + 1;
end

t = toc;

if opts.validation == 1
    loss = nneval(nn, loss, train_x, train_y, val_x, val_y);
    str_perf = sprintf('; Full-batch train mse = %f, val mse = %f',
loss.train.e(end), loss.val.e(end));
else
    loss = nneval(nn, loss, train_x, train_y);
    str_perf = sprintf('; Full-batch train err = %f', loss.train.e(end));
end
if ishandle(fhandle)
    nnupdatefigures(nn, fhandle, loss, opts, i);
end

disp(['epoch ' num2str(i) '/' num2str(opts.numepochs) '. Took ' num2str(t) '  seconds' '. Mini-batch mean squared error on training set is ' num2str(mean(L((n-numbatches):(n-1)))) str_perf]);
nn.learningRate = nn.learningRate * nn.scaling_learningRate;
end

Can anyone let me know, is the training for the NN like this enable it to do the segmentation work? or how should I modify the code to train the NN so that it can generate the output/result as the image matrix in 150x10201 form?

Thank you so much..

FiReTiTi FiReTiTi · Accepted Answer · 2015-10-08T05:16:30

Your inputs are TOO big. You should try to work with smaller patches from 19x19 to maximum 30x30 (which already represent 900 neurons into the input layer).

Then comes your main problem: you only have 150 images! And when you train a NN, you need at least three times more training instances than weights into your NN. So be very careful to the architecture you choose.

A CNN may be more adapted to your problem.

Training neural network for image segmentation

1 Answers