Generating multivariate normally distributed random numbers in Matlab

Question

This question is about the use of the covariance matrix in the multidimensional normal distribution:

I want to generate multi-dimensional random numbers x in Matlab with a given mean mu and covariance matrix Sigma. Assuming Z is a standard normally distributed random number (e.g. generated using randn), what is the correct code:

x = mu + chol(Sigma) * Z

or

x = mu + Sigma ^ 0.5 * Z

?

I am not sure about the use of the covariance matrix in the definition of the multidimensional normal distribution – whether the determinant in the denominator is of the square root or the Cholesky factor...

It's the first option, but I think you need to include a complex transpose. See here. Alternatively, use Matlab's mvnrnd function, which accepts sigma and does the Chlesky decomposition internally — Luis Mendo
@LuisMendo, afaics ^0.5 or sqrtm should work as well. What is the advantage of chol? I guess numeric precision of recovering Sigma? — A. Donda
@A.Donda Sorry, I misread Sigma^0.5. Actually I'm not sure about that. What I do know is that the first option is correct. Maybe the second is too — Luis Mendo

A. Donda A. Donda · Accepted Answer · 2015-01-23T18:01:21

If by definition you refer to the density of the multivariate normal distribution:

it contains neither the Cholesky decomposition nor the matrix square root of Σ, but its inverse and the scalar square root of its determinant.

But for numerically generating random numbers from this distribution, the density is not helpful. It is not even the most general description of the multivariate normal distribution, since the density formula makes only sense for positive definite matrices Σ, while the distribution is also defined if there are zero eigenvalues – that just means that the variance is 0 in the direction of the respective eigenvector.

Your question follows the approach to start from standard multivariate normally distributed random numbers Z as produced by randn, and then apply a linear transformation. Assuming that mu is a p-dimensional row vector we want an nxp-dimensional random matrix (each row one observation, each column one variable):

Z = randn(n, p);
x = mu + Z * A;

We need a matrix A such that the covariance of x is Sigma. Since the covariance of Z is the identity matrix, the covariance of x is given by A' * A. A solution to this is given by the Cholesky decomposition, so the natural choice is

A = chol(Sigma);

where A is an upper triangular matrix.

However, we can also search for a Hermitian solution, A' = A, and then A' * A becomes A^2, the matrix square. A solution to this is given by a matrix square root, which is computed by replacing each eigenvalue of Sigma by its square root (or its negative); in general there are 2ⁿ possible solutions for n positive eigenvalues. The Matlab function sqrtm returns the principal matrix square root, which is the unique nonnegative-definite solution. Therefore,

A = sqrtm(Sigma)

works also. A ^ 0.5 should in principle do the same.

Simulations using this code

p = 10;
n = 1000;

nr = 1000;
cp = nan(nr, 1);
sp = nan(nr, 1);
pp = nan(nr, 1);

for i = 1 : nr
    x = randn(n, p);
    Sigma = cov(x);

    cS = chol(Sigma);
    cp(i) = norm(cS' * cS - Sigma);

    sS = sqrtm(Sigma);
    sp(i) = norm(sS' * sS - Sigma);

    pS = Sigma ^ 0.5;
    pp(i) = norm(pS' * pS - Sigma);
end    

mean([cp sp pp])

yield that chol is more precise than the two other methods, and profiling shows that it is also much faster, for both p = 10 and p = 100.

The Cholesky decomposition does however have the disadvantage that it is only defined for positive-definite Σ, while the requirement of the matrix square root is merely that Σ is nonnegative-definite (sqrtm returns a warning for a singular input, but returns a valid result).

Generating multivariate normally distributed random numbers in Matlab

1 Answers