Problem 4: Producing normal vectors
Published:
The idea is to produce multivariate normal random vectors from univariate standard normal random numbers. For this, let $Z$ be a $\mathcal{N}(0,I_n)$ a random vector (here $I_n$ denotes the $n\times n$ identity matrix). Given a vector $\mu$ and a symmetric real matrix $\Sigma$ (to be the mean and covariance parameters of the multivariate normal vector), consider the Cholesky decomposition of $\Sigma$, given by $\Sigma = LL^T$. Prove that $X = \mu +LZ$ is distributes as $\mathcal{N}(\mu,\Sigma)$.
Solution:
First we compute the expected value of $X$. Since $\mu$ and $L$ are not random and the expected value is linear, we have that
\[\mathbb{E}(X) = \mu + L\mathbb{E}(Z) =\mu,\]as the expected value of $Z$ is the $n$-th dimensional zero vector. On the other hand, for the covariance of $X$ we obtain
\[\newcommand{\cov}{\mathrm{cov}} \cov(X) = \cov(\mu)+\cov(LZ)=L\cov(Z)L^T = LL^T =\Sigma\]where the covariance of $\mu$ is zero as it is not a random quantity. To check that $X$ is multivariate normal, we take linear combinations of its components and prove they give normally distributed numbers. Introducing index notation, we have that
\[X_j = \mu_j+ \sum_{k=1}^n L_{jk}Z_k.\]Given an $n$-dimensional vector $a=(a_1,\dots,a_n)$, we have that
\[a\cdot X = \sum_{j=1}^n a_j(\mu_j+ \sum_{k=1}^n L_{jk}Z_k) = a\cdot\mu+\sum_{k=1}^n \left( \sum_{j=1}^n a_j L_{jk}\right)Z_k.\]From the last equation we can see that $a\cdot X$ is nothing but a sum of normally distributed random variables, hence, it is normally distributed as well, proving the result.
Leave a Comment