Image Reconstruction using ICA vs PCA

Question

I am currently experimenting using the PCA and ICA algorithms. I have quite comfortable using PCA now to create set of eigenfaces, select the optimal eigenfaces and reconstruct an image, and I am also able to use PCA to perform face hallucination. I wish to do the same with ICA, but I am unsure about how to do this.

The steps I have taken so far are this, perform PCA, find orthogonal eigenvectors which represents the eigenfaces. Currently experimenting on Yale ORL and PIE databases

I then perform fastica on these eigenvectors like so

%Perform ICA on eigenvectors transposed
%icasig = source matrix
%A denotes unknown mixing matrix
%W denotes unmixing matrix - inverse of A
%Variable R are the orthogonal eigenvectors
[icasig A W] = fastica(R');

I then implemented ICA architecture 1 to find the ICA coefficients. Algorithm for ICA architecture 1 can be found here

http://www.sciencedirect.com/science/article/pii/S1077314203000778

Just scroll down to section 2.3.1

%centeredMatrix is the image matrix of my training set minus the mean so here
%I take out one of the mean centered images
x = centeredMatrix(:,1);

%Calculate Independent basis images

U = W * R';

%Calculate PCA Coefficients
C = x' * R;

%Calculate ICA Coefficients
B = C * inv(W);

% plot first 16 PCA faces
figure; hold on;
for i=1:16
  subplot(4,4,i);
  comp = reshape(R(:,i), width, height);
  imagesc(comp),colormap('gray')
  title(sprintf('PCAFace #%i', i));
end

% plot first 16 ICAFaces
figure; hold on;
for i=1:16
  subplot(4,4,i);
  comp = reshape(U(i,:), width, height);
  imshow(comp);
  title(sprintf('ICAFace #%i', i));
end

I know I can do PCA reconstruction from this

%-----------------PCA Reconstruction----------

%Eigenvectors * pcaCoefficients + mean image
pcaRecon = (R * C') + mu;
pcaRecon = uint8(pcaRecon);
pcaRecon = reshape(pcaRecon,width,height);
figure, imshow(pcaRecon), title('PCA Recon');

But I am unsure as to how I would go about doing an ICA reconstruction. By implementing ICA architecture 1, I have been able to perform a reconstruction, by doing this

%-----------------ICA Reconstruction----------

icaRecon = (icasig' * B') + mu;
icaRecon = uint8(icaRecon);
icaRecon = reshape(icaRecon,width,height);
figure, imshow(icaRecon), title('ICA Recon');

but I am unsure if this is a good method for reconstruction, it doesn't seem to be very effective compared to PCA. Above code was tested on Yale database which has 165 images. With PCA I can use about 80 eigenfaces and still achieve a very high quality image whereas with ICA method here, I have to use around 105 - 110 to get similar quality image. Same with ORL database which contains 400 images, can perform PCA, gen eigenfaces, and then use around 120 eigenfaces for reconstruction and get very high quality image, whereas with ICA, need around 380 of the components to get a similar quality reconstruction

I am still new to ICA, so there is still a lot to learn, but couple of questions I would like answered if possible are

1: Is this ICA method I am using to do reconstruction a good method? Or is there a better method you would recommend. I have

2: Why do I need to use so many components with ICA to do reconstruction, I am guessing this is a problem with my method?

Would be very grateful if someone could point me the right direction or explain to me a better method for doing reconstruction.

Thanks in advance

Edit:

Example results from ORL database, comparing original with PCA and ICA reconstruction. PCA reconstruction was done using 200 eigenfaces, ICA reconstruction done using 200 ICs. As you can see the ICA reconstruction is pretty poor in comparison

enter image description here

jez jez · Accepted Answer · 2013-11-21T23:59:49

Why do I need to use so many components with ICA to do reconstruction, I am guessing this is a problem with my method?

I haven't checked your code in detail, but one possible answer to this is that PCA components come out ordered (the one that explains the largest amount of variance in the input data set comes out first) whereas typically ICA components come out in arbitrary order. So if you're reconstructing from only a subset of n components, in PCA it's easy: pick the first n. In ICA you'd need some other criterion for selecting which n components are, for want of a better word, the "faciest" ones.

Image Reconstruction using ICA vs PCA

2 Answers