Feature extraction with Gabor filters

Question

I am currently working on feature extraction system with use of Gabor filters in Matlab. For Gabor filters and convolution of image I am using code found here https://www.mathworks.com/matlabcentral/fileexchange/44630-gabor-feature-extraction ,just slightly adjusted for my own use. Faces for this system are obtained from AT&T face database http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html I use 9 subjects from the database. Each subject has 10 images belonging to him/her. With use of Viola Jones I crop the face image and resize it to 128x128px.

This function creates Gabor filter bank. Where u=5, v=8, m,n = 39. So resulting gaborArray consists of 40 different Gabor filters of 5 scales and 8 orientations. I assume these 2 functions are working as intended since they are from approved external source.

function gaborArray = gaborFilterBank(u,v,m,n)
gaborArray = cell(u,v);
fmax = 0.25;
gama = sqrt(2);
eta = sqrt(2);

for i = 1:u    
    fu = fmax/((sqrt(2))^(i-1));
    alpha = fu/gama;
    beta = fu/eta;    
    for j = 1:v
        tetav = ((j-1)/v)*pi;
        gFilter = zeros(m,n);

        for x = 1:m
            for y = 1:n
                xprime = (x-((m+1)/2))*cos(tetav)+(y-((n+1)/2))*sin(tetav);
                yprime = -(x-((m+1)/2))*sin(tetav)+(y-((n+1)/2))*cos(tetav);
                gFilter(x,y) = (fu^2/(pi*gama*eta))*exp(-((alpha^2)*(xprime^2)+(beta^2)*(yprime^2)))*exp(1i*2*pi*fu*xprime);
            end
        end
        gaborArray{i,j} = gFilter;        
    end
end

This is 2nd part where I load my database and convolve each image with Gabor Bank stored in gaborArray which was obtain from previous function. Since I work with 9 subjects x 10 images result is of cellsize 90 x gaborAbs + 1 for class (1-9)

function [result]  = feature(gaborArray)
faceDatabase = imageSet(database,'recursive');    %database = folder
numberOfFoldersOverall = size(faceDatabase,2);
numberOfPhotosPF = 10;
cellSize = numberOfFoldersOverall * numberOfPhotosPF;
allFeatures = cell(cellSize,1);

[u,v] = size(gaborArray);
gaborResult = cell(u,v);
d1 = 4;
d2 = 4;
count = 1;

for m=1:numberOfFoldersOverall
    for n=1:numberOfPhotosPF
        I= (read(faceDatabase(m), n));
        I = double(I);
        featureVector = [];

        for i = 1:u
            for j = 1:v
                gaborResult{i,j} = imfilter(I, gaborArray{i,j});
            end
        end
        for i = 1:u
            for j = 1:v
                gaborAbs = abs(gaborResult{i,j});
                gaborAbs = downsample(gaborAbs,d1); 
                gaborAbs = downsample(gaborAbs.',d2);
                gaborAbs = gaborAbs(:);

                gaborAbs = (gaborAbs-mean(gaborAbs))/std(gaborAbs,1);
                featureVector =  [featureVector; gaborAbs];                                
             end
         end
         featureVector = [featureVector; m];
         allFeatures{count} = featureVector;
         count = count + 1;
    end
end
result = allFeatures;

This is the part where I probably made mistake in implementation.With help from Gabor feature extraction I made few changes to previous function to compute Local Energy

gaborAbs = gaborResult{i,j};     %no longer abs(gaborResult{i,j})
gaborAbs = downsample(gaborAbs,d1); 
gaborAbs = downsample(gaborAbs.',d2);
gaborAbs = gaborAbs(:);
gaborAbs = sum(gaborAbs).^2;    %sum of squared gaborAbs

and to compute Mean Amplitude

gaborAbs = abs(gaborResult{i,j});                         
gaborAbs = downsample(gaborAbs,d1);                       
gaborAbs = downsample(gaborAbs.',d2);
gaborAbs = gaborAbs(:);
gaborAbs = sum(gaborAbs);       % sum of abs gaborAbs

Both vektors of Local energy and Mean Amplitude have length of 41 where last index is class of subject. However when either is used in my face recognition Knn based system I end up with 10% recognition rate in some cases.

I tried to plot Local energy feature vektor using andrewsplot in matlab with this as result. Each line represents 1 subject so there are together 90 lines, where lines with same color belong to same class(same person on image). As we can see they are stacked on top of each other with little to no difference

SO finally, and I am sorry for this lenghtly description I ask of you.

Have I made mistakes when computing Local Energy or Mean amplitude?
Is Andrewsplot fine for representing such data?
If yes am I interpreting it good as all feature vektors are basicly almost identical since there is little to no difference in graph?(I assume that images belonging to same person should be stacked but differ from others. Here everything STACKS!)
Or is there possibility that these features are simply not good for face recognition? Thank you in Advance for any tips or examples. With best regards Yondaru.

Cris Luengo Cris Luengo · Accepted Answer · 2018-03-06T03:07:32

Here is my first thought. This is not going to be a complete answer, but maybe you can take it form here.

The magnitude of the Gabor filter result gives a fairly smooth result: it is the amount of the given spatial frequency present in the local neighborhood. But if you look at the real and imaginary parts separately, you'll see a high-frequency signal (equal to the frequency of the filter itself). I encourage you to display some of these results as images to get a feeling for it.

Downsampling the smooth magnitude is OK. It's smooth, you'll get representative results for the neighborhood. But downsampling the real and imaginary components is dangerous: you'll probably see aliasing, or at best a meaningless quantity. Of course, these two components together still have a meaningful magnitude, but the phase is meaningless.

In any case, since energy is the square of the norm, you don't really need hat downsampled phase. Just square and sum the magnitude image.

Lastly, if you are summing anyway, why downsample? There is no advantage that I can see.

In terms of coding: why save in gaborResult all filtering results? You might as well loop over all u,v once, apply the filter, and directly compute your sum magnitude for that filter.

Feature extraction with Gabor filters

1 Answers