Multivariate Normal distribution

Story

This is a generalization of the univariate Normal distribution.

Example

Finch beaks are measured for beak depth and beak length. The resulting distribution of depths and length is Normal. In this case, the Normal is bivariate, with \(\boldsymbol{\mu} = (\mu_d, \mu_l)\) and the covariance matrix is

\[\begin{split}\begin{align} \mathsf{\Sigma} = \begin{pmatrix}\sigma_\mathrm{d}^2 & \sigma_\mathrm{dl} \\ \sigma_\mathrm{dl} & \sigma_\mathrm{l}^2\end{pmatrix}. \end{align}\end{split}\]

Parameters

There is one vector-valued parameter, \(\boldsymbol{\mu}\), and a matrix-valued parameter, \(\mathsf{\Sigma}\), which are location and scale parameters respectively. The matrix scale parameter is referred to as a covariance matrix. The covariance matrix is symmetric and strictly positive definite.

Support

The \(K\)-variate Normal distribution is supported on \(\mathbb{R}^K\).

Probability density function

\[\begin{align} f(\mathbf{y};\boldsymbol{\mu}, \mathsf{\Sigma}) = \frac{1}{\sqrt{(2\pi)^K \mathrm{det}\mathsf{\Sigma}}}\,\exp\left[-\frac{1}{2}(\mathbf{y} - \boldsymbol{\mu})^\mathsf{T} \cdot \mathsf{\Sigma}^{-1} \cdot (\mathbf{y} - \boldsymbol{\mu})\right]. \end{align}\]

Cumulative distribution function

There is no analytic expression for the CDF.

Moments

Mean of \(y_i\): \(\mu_i\)

Variance of \(y_i\): \(\Sigma_{ii}\)

Covariance of \(y_i, y_j\) with \(j\ne i\): \(\Sigma_{ij}\)

Usage

The usage below assumes that mu is a length \(K\) array, Sigma is a \(K\times K\) symmetric positive definite matrix, and L is a \(K\times K\) lower-triangular matrix with strictly positive values on the diagonal that is a Cholesky factor.

Package	Syntax
NumPy	`rng.multivariate_normal(mu, Sigma)`
NumPy Cholesky	`rng.multivariate_normal(mu, np.dot(L, L.T))`
SciPy	`scipy.stats.multivariate_normal(mu, Sigma)`
SciPy Cholesky	`scipy.stats.multivariate_normal(mu, np.dot(L, L.T))`
Distributions.jl	`MvNormal(mu, Sigma)`
Distributions.jl Cholesky	`MvNormal(mu, L * L')`
Stan	`multi_normal(mu, Sigma)`
Stan Cholesky	`multi_normal_cholesky(mu, L)`

Notes

The covariance matrix may also be written as \(\mathsf{\Sigma} = \mathsf{S} \cdot \mathsf{C} \cdot \mathsf{S}\), where \(\mathsf{S} = \sqrt{\mathrm{diag}(\mathsf{\Sigma})}\), and entry \(i, j\) in the correlation matrix \(\mathsf{C}\) is \(C_{ij} = \sigma_{ij}/\sigma_i\sigma_j\).
Because \(\mathsf{\Sigma}\) is symmetric and strictly positive definite, it can be uniquely defined in terms of its Cholesky decomposition, \(\mathsf{L}\), which satisfies \(\mathsf{\Sigma} = \mathsf{L}\cdot\mathsf{L}^\mathsf{T}\).