Covariance matrix computation

8
votes

Input : random vector X=xi, i=1..n.
vector of means for X=meanxi, i=1..n
Output : covariance matrix Sigma (n*n).
Computation :
1) find all cov(xi,xj)= 1/n * (xi-meanxi) * (xj-meanxj), i,j=1..n
2) Sigma(i,j)=cov(xi,xj), symmetric matrix.
Is this algorithm correct and has no side-effects?

1
algorithmmatrixstatisticscovariance
The problem statement is not very clear. Do you really have one single vector as input? Do the xi all have the same mean? Why would you divide by (n-1) when calculating the mean? - Henrik
In theory i have a lot of them (X is actually some process X(t)) where t is [0..k], but during modeling i'm interested only in case k=kmax, that's why i get single vector X(kmax)=X which consists of r numbers. n-1 is correction, doesn't affect much. About means - they are different as i see now. - Singularity
I'm voting to close this question as off-topic because it's a math verification question, not a programming question within the scope of the help center. - TylerH

1 Answers

5
votes

Each xi should be a vector (random variable) with it's own variance and mean.

Covariance matrix is symmetric, so you just need to compute one half of it (and copy the rest) and has variance of xi at main diagonal. 

 S = ...// your symmetric matrix n*n
 for(int i=0; i<n;i++)
   S(i,i) = var(xi);
   for(j = i+1; j<n; j++)
     S(i,j) = cov(xi, xj);
     S(j,i) = S(i,j);
   end
 end

where variance (var) of xi: 

v = 0;
for(int i = 0; i<xi.Count; i++)
  v += (xi(i) - mean(xi))^2;
end
v = v / xi.Count;

and covariance (cov)

cov(xi, xj) = r(xi,xj) * sqrt(var(xi)) * sqrt(var(xj))

where r(xi, xj) is Pearson product-moment correlation coefficient

EDIT
or, since cov(X, Y) = E(X*Y) - E(X)*E(Y)

cov(xi, xj) = mean(xi.*xj) - mean(xi)*mean(xj);

where .* is Matlab-like element-wise multiplication.
So if x = [x1, x2], y = [y1, y2] then z = x.*y = [x1*y1, x2*y2];