Normal distribution

---

Story

Any quantity that emerges as the sum of a large number of subprocesses tends to be Normally distributed provided none of the subprocesses is very broadly distributed.

---

Example

We measure the length of many C. elegans eggs. The lengths are Normally distributed. Many biological measurements, like the height of people, are (approximately) Normally distributed. Many processes contribute to setting the length of an egg or the height of a person.

---

Parameters

The Normal distribution has two parameters, the location parameter \(\mu\), which determines the location of its peak, and the scale parameter \(\sigma\), which is strictly positive (the \(\sigma \to 0\) limit defines a Dirac delta function) and determines the width of the peak.

These parameters are commonly referred to as the mean and standard deviation, respectively. Those terms are widely used in other contexts, such as for point estimates of moments of an arbitrary distribution, so we avoid using those terms to avoid confusion.

---

Support

The Normal distribution is supported on the set of real numbers.

---

Probability density function

\[\begin{align} f(y;\mu, \sigma) = \frac{1}{\sqrt{2\pi\sigma^2}}\,\mathrm{e}^{-(y-\mu)^2/2\sigma^2}. \end{align}\]

---

Cumulative distribution function

\[\begin{align} F(y;\mu, \sigma) =\frac{1}{2}\left(1 + \text{erf}\left(\frac{y - \mu}{\sigma\sqrt{2}}\right)\right), \end{align}\]

where \(\text{erf}(x)\) denotes the error function.

---

Moments

Mean: \(\mu\)

Variance: \(\sigma^2\)

---

Usage

Package	Syntax
NumPy	rng.normal(mu, sigma)
SciPy	scipy.stats.norm(mu, sigma)
Distributions.jl	Normal(mu, sigma)
Stan	normal(mu, sigma)

---

Related distributions

• The Binomial distribution, parametrized by \(N\) and \(\theta\) is approximated Normal with location parameter \(N\theta\) and scale parameter \(\sqrt{N\theta(1-\theta)}\) for large \(N\) and \(\theta\) not too close to zero or one. Although the Binomial distribution is discrete, for large \(N\) we approximate it as continuous, allowing the Normal approximation.

• It follows that the Poisson distribution with parameter \(\lambda\) is approximately normal with location parameter \(\lambda\) and scale parameter \(\sqrt{\lambda}\) for large \(\lambda\).

• The Student-t distribution, parametrized by location and scale parameters \(\mu\) and \(\sigma\), respectively, and degrees of freedom \(\nu\) becomes Normal (with the same location and scale parameters) in the limit of \(\nu\to\infty\).

• The Gamma distribution, parametrized by shape and inverse scale parameters \(\alpha\) and \(\beta\), respectively, is approximately Normal with respective location and scale parameters \(\alpha/\beta\) and \(\sqrt{\alpha}/\beta\) in the limit of large \(\alpha\).

• Any univariate continuous distribution is approximately Normal very close to a peak. Imagine a distribution with a PDF given by \(f(y;\theta)\), where \(\theta\) represents the set of parameters for the distribution. If we expand the log PDF to second order in a Taylor series about a point \(y^*\) where the PDF is maximal, we get

  \[\begin{align} \ln f(y;\theta) \approx \ln f(y^*;\theta) + \left.\frac{1}{2}\,\frac{\mathrm{d}^2 \ln f(y;\theta)}{\mathrm{d}y^2}\right|_{y=y^*}\,(y-y^*)^2. \end{align}\]

  This is the log PDF that is that of a Normal distribution. The location parameter of the approximate Normal distribution is \(y^*\) (the location of the peak) and the scale parameter is

  \[\begin{align} \left(\left.-\frac{\mathrm{d}^2 \ln f( y;\theta)}{\mathrm{d}y^2}\right|_{y=y^*}\right)^{-1/2}. \end{align}\]

  Note, though, that this may be a very poor approximation as we move even a small distance from the peak.

---

Notes

• The Normal distribution is often referred to as the Gaussian distribution, particularly in the physical sciences, named for one of its discoverers, Carl Gauss.

---

PDF and CDF plots

---

Links

• Wikipedia

• Numpy

• Scipy

• Distributions.jl

• Stan