Numpy.eig and the percentage of variance in PCA

Question

So I can use linalg.eig or linalg.svd to compute the PCA. Each one returns different Principal Components/Eigenvectors and Eigenvalues when they're fed the same data (I'm currently using the Iris dataset).

Looking here or any other tutorial with the PCA applied to the Iris dataset, I'll find that the Eigenvalues are [2.9108 0.9212 0.1474 0.0206]. The eig method gives me a different set of eigenvalues/vectors to work with which I don't mind, except that these eigenvalues, once summed, equal the number of dimensions (4) and can be used to find how much each component contributes to the total variance.

Taking the eigenvalues returned by linalg.eig I can't do that. For example, the values returned are [9206.53059607 314.10307292 12.03601935 3.53031167]. The proportion of variance in this case would be [0.96542969 0.03293797 0.00126214 0.0003702]. This other page says that ("The proportion of the variation explained by a component is just its eigenvalue divided by the sum of the eigenvalues.")

Since the variance explained by each dimension should be constant (I think), these proportions are wrong. So, if I use the values returned by svd(), which are the values used in all tutorials, I can get the correct percentage of variation from each dimension, but I'm wondering why the values returned by eig can't be used like that.

I assume the results returned are still a valid way to project the variables, so is there a way to transform them so that I can get the correct proportion of variance explained by each variable? In other words, can I use the eig method and still have the proportion of variance for each variable? Additionally, could this mapping be done only in the eigenvalues so that I can have both the real eigenvalues and the normalized ones?

Sorry for the long writeup btw. Here's a (::) for having gotten this far. Assuming you didn't just read this line.

@S.Lott I tried posting there once and they said the site was only for real advanced math and stuff so I'd rather not go there again unless I really have to. — pcapcapcapca
Since this isn't programming (no code), you are less likely to get help here. A single question with snarky comments doesn't mean much. I'd start a search there for related questions before waiting around for an answer here. It's important to read a lot of questions to see how the questions are asked so you can fit in better. Example. "Sorry for the long writeup" is lame. If it's long, you can fix it to be short and to be point. — S.Lott
@S.Lott my question is somewhat specific to python. i can ask there (i'm typing the question on the other tab) but i'm pretty confident they'll just tell me they can't help because of that. plus, i wrote a lot of text because often you get people asking to clarify what you mean, if the other questions i read are any indication. i'm pretty sure you're mad because you just read the last line and got no cookie — pcapcapcapca

las3rjock las3rjock · Accepted Answer · 2011-01-28T04:47:46

Taking Doug's answer to your previous question and implementing the following two functions, I get the output shown below:

def pca_eig(orig_data):
    data = array(orig_data)
    data = (data - data.mean(axis=0)) / data.std(axis=0)
    C = corrcoef(data, rowvar=0)
    w, v = linalg.eig(C)
    print "Using numpy.linalg.eig"
    print w
    print v

def pca_svd(orig_data):
    data = array(orig_data)
    data = (data - data.mean(axis=0)) / data.std(axis=0)
    C = corrcoef(data, rowvar=0)
    u, s, v = linalg.svd(C)
    print "Using numpy.linalg.svd"
    print u
    print s
    print v

Output:

Using numpy.linalg.eig
[ 2.91081808  0.92122093  0.14735328  0.02060771]
[[ 0.52237162 -0.37231836 -0.72101681  0.26199559]
 [-0.26335492 -0.92555649  0.24203288 -0.12413481]
 [ 0.58125401 -0.02109478  0.14089226 -0.80115427]
 [ 0.56561105 -0.06541577  0.6338014   0.52354627]]

Using numpy.linalg.svd
[[-0.52237162 -0.37231836  0.72101681  0.26199559]
 [ 0.26335492 -0.92555649 -0.24203288 -0.12413481]
 [-0.58125401 -0.02109478 -0.14089226 -0.80115427]
 [-0.56561105 -0.06541577 -0.6338014   0.52354627]]
[ 2.91081808  0.92122093  0.14735328  0.02060771]
[[-0.52237162  0.26335492 -0.58125401 -0.56561105]
 [-0.37231836 -0.92555649 -0.02109478 -0.06541577]
 [ 0.72101681 -0.24203288 -0.14089226 -0.6338014 ]
 [ 0.26199559 -0.12413481 -0.80115427  0.52354627]]

In both cases, I get the desired eigenvalues.

Numpy.eig and the percentage of variance in PCA

4 Answers