AMH - g_ud.3

Definition

A continuous random variable $X$ has the with mean $\mu$ and variance $\sigma^2$ if the pdf of $X$ is $f(x) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$

We say $X$ has the Normal $(\mu, \sigma^2)$ distribution.

There is no closed from cdf for a normal distribution.

Claim: The function

f (x) = \frac{1}{\sqrt{2 π σ^{2}}} e^{- \frac{(x - μ)^{2}}{2 σ^{2}}}

$f(x) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$ is a pdf.

▼ Proof:

Since

f (x) > 0

$f(x) > 0$ for all

x,

$x,$ the only thing that needs to be shown is that

\int_{- \infty}^{\infty} f (x) d x = 1.

$\int_{-\infty}^\infty f(x) dx = 1.$ First, substitute

x - μ

$x - \mu$ for

x

$x$ to eliminate the

μ

$\mu$ in the exponent.

\int_{- \infty}^{\infty} \frac{1}{\sqrt{2 π σ^{2}}} e^{\frac{- (x - μ)^{2}}{2 σ^{2}}} d x = \int_{- \infty}^{\infty} \frac{1}{\sqrt{2 σ^{2}}} e^{\frac{- x^{2}}{2 σ^{2}}} d x

$\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi\sigma^2}}e^{\frac{-(x-\mu)^2}{2\sigma^2}} dx = \int_{-\infty}^\infty \frac{1}{\sqrt{2\sigma^2}}e^{\frac{-x^2}{2\sigma^2}} dx$ It is easier to compute the square of the integral rather than the integral itself.

\begin{matrix} {(\int_{- \infty}^{\infty} \frac{1}{\sqrt{2 π σ^{2}}} e^{- \frac{x^{2}}{2 σ^{2}}} d x)}^{2} & = (\int_{- \infty}^{\infty} \frac{1}{\sqrt{2 π σ^{2}}} e^{- \frac{x^{2}}{2 σ^{2}}} d x) (\int_{- \infty}^{\infty} \frac{1}{\sqrt{2 π σ^{2}}} e^{- \frac{x^{2}}{2 σ^{2}}} d x) = (\int_{- \infty}^{\infty} \frac{1}{\sqrt{2 π σ^{2}}} e^{- \frac{x^{2}}{2 σ^{2}}} d x) (\int_{- \infty}^{\infty} \frac{1}{\sqrt{2 π σ^{2}}} e^{- \frac{y^{2}}{2 σ^{2}}} d y) = \frac{1}{2 π σ^{2}} \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} e^{- \frac{x^{2}}{2 σ^{2}}} e^{- \frac{y^{2}}{2 σ^{2}}} d x d y = \frac{1}{2 π σ^{2}} \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} e^{- \frac{x^{2} + y^{2}}{2 σ^{2}}} d x d y \end{matrix}

$\begin{align} \left( \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{x^2}{2\sigma^2}} dx\right)^2 & = \left( \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{x^2}{2\sigma^2}} dx\right)\left( \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{x^2}{2\sigma^2}} dx\right) \\ & = \left( \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{x^2}{2\sigma^2}} dx\right)\left( \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}} dy\right) \\ & = \frac{1}{2\pi\sigma^2}\int_{-\infty}^\infty \int_{-\infty}^\infty e^{-\frac{x^2}{2\sigma^2}}e^{-\frac{y^2}{2\sigma^2}} dxdy \\ & = \frac{1}{2\pi\sigma^2}\int_{-\infty}^\infty \int_{-\infty}^\infty e^{-\frac{x^2+y^2}{2\sigma^2}} dxdy \\ \end{align}$ Now use polar coordinates. Substitute

r^{2}

$r^2$ for

x^{2} + y^{2}

$x^2+y^2$ and

r d r d θ

$rdrd\theta$ for

d x d y .

$dxdy.$

\begin{matrix} \frac{1}{2 π σ^{2}} \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} e^{- \frac{x^{2} + y^{2}}{2 σ^{2}}} d x d y & = \frac{1}{2 π σ^{2}} \int_{0}^{2 π} \int_{0}^{\infty} e^{- \frac{r^{2}}{2 σ^{2}}} r d r d θ = \frac{1}{2 π σ^{2}} \int_{0}^{2 π} \int_{0}^{\infty} r e^{- \frac{r^{2}}{2 σ^{2}}} d r d θ \end{matrix}

$\begin{align} \frac{1}{2\pi\sigma^2}\int_{-\infty}^\infty \int_{-\infty}^\infty e^{-\frac{x^2+y^2}{2\sigma^2}} dxdy & = \frac{1}{2\pi\sigma^2}\int_0^{2\pi} \int_0^\infty e^{-\frac{r^2}{2\sigma^2}} rdrd\theta \\ & = \frac{1}{2\pi\sigma^2}\int_0^{2\pi} \int_0^\infty re^{-\frac{r^2}{2\sigma^2}} drd\theta \end{align}$ Substitute

$u = r^2,$ so

$du = 2rdr.$

$\begin{align} \frac{1}{2\pi\sigma^2}\int_0^{2\pi} \int_0^\infty re^{\frac{-r^2}{2\sigma^2}} drd\theta & = \frac{1}{2\pi\sigma^2}\int_0^{2\pi} \int_0^\infty \frac{1}{2}e^{\frac{-u}{2\sigma^2}} dud\theta \\ & = \frac{1}{2\pi\sigma^2}\int_0^{2\pi} \left.\frac{-2\sigma^2}{2}e^{-\frac{u}{2\sigma^2}}\right|_{u=0}^\infty d\theta \\ & = \frac{1}{2\pi\sigma^2}\int_0^{2\pi} \sigma^2 d\theta \\ & = \frac{1}{2\pi\sigma^2}2\pi\sigma^2 \\ & = 1 \end{align}$ Since

$(\int_{-\infty}^\infty f(x) dx)^2 = 1$ and

$f(x) > 0,$

$\int_{-\infty}^\infty f(x) = 1.$

Claim: The expected value of

$X$ is

$E[X] = \mu.$

▼ Proof:

The result can be found by direct computation.

$\begin{align} E[X] & = \int_{-\infty}^\infty \frac{x}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} dx \\ & = \int_{-\infty}^\infty \frac{y+\mu}{\sqrt{2\pi\sigma^2}}e^{\frac{-y^2}{2\sigma^2}} dy \end{align}$ where the last line follows by substituting

$y+\mu$ for

$x.$ Splitting the integral over the

$y+\mu$ we get

$\begin{align} \int_{-\infty}^\infty \frac{y+\mu}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}} dy & = \int_{-\infty}^\infty \frac{y}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}}dy + \int_{-\infty}^\infty \frac{\mu}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}}dy \\ & = \int_{-\infty}^\infty \frac{y}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}}dy + \mu\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}}dy \end{align}$ Now we compute each integral in turn. The left integral is

$\begin{align} \int_{-\infty}^\infty \frac{y}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}}dy & = \int_{-\infty}^0 \frac{y}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}}dy + \int_0^\infty \frac{y}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}}dy \\ & = -\int_{0}^\infty \frac{y}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}}dy + \int_0^\infty \frac{y}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}}dy \\ & = 0 \end{align}$ where the second line follows from a substitution of

$-y$ for

$y.$

The second integral is an integral of a pdf of a normal random variable, so it is equal to

$1.$

$\begin{align} \mu\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{y^2}{2\sigma^2}}dy & = \mu \cdot 1 \\ & = \mu \end{align}$

: The variance of $X$ is $\text{Var}(X) = \sigma^2.$

The proof of this fact is reserved for later.

Standard Normal

A has mean $0$ and variance $1.$ The pdf for a standard normal random variable is $f(x) = \frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}$

: A normal random variable $X$ with mean $\mu$ and standard deviation $\sigma$ can be transformed to a standard normal random variable by subtracting the mean and dividing the standard deviation: $\frac{X - \mu}{\sigma}$

▼ Proof:

Let

$Y = \frac{X - \mu}{\sigma},$ and let

$F_X(t)$ and

$F_Y(t)$ be the cdf's of

$X$ and

$Y,$ respectively. By computation, we can write

$F_Y(t)$ in terms of

$F_X(t).$

$\begin{align} F_Y(t) & = P\left(\frac{X-\mu}{\sigma} \leq t\right) \\ & = P(X \leq \sigma t + \mu) \\ & = F_X(\sigma t + \mu) \end{align}$ Take the derivative of both sides to convert the cdf's to pdf's.

$\begin{align} & \frac{d}{dt}F_Y(t) = \frac{d}{dt}F_X(\sigma t + \mu) \Rightarrow & f_Y(t) = \sigma f_X(\sigma t + \mu) \end{align}$ So, the pdf of

$Y$ is

$\begin{align} f_Y(t) & = \sigma \cdot \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{((\sigma t + \mu) - \mu)^2}{2\sigma^2}} \\ & = \frac{1}{\sqrt{2\pi}}e^{-\frac{t^2}{2}} \end{align}$ which shows

$Y,$ or

$\frac{X-\mu}{\sigma},$ has the standard normal distribution.

Bell Curve

Normal random variables take the shape of a bell curve. The highest point on the curve is the mean. The curve is always symmetric about the mean. The curve will be narrow or wide depending on the standard deviation.

This graph shows the pdf of a standard normal random variable.

This graph shows the pdf of a Normal

$(0,2)$ random variable. The standard normal is in gray for reference. The higher variance makes the graph wider and flatter.

This graph shows the pdf of a Normal

$(1,1)$ random variable. The standard normal is in gray for reference. Raising the mean shifts the graph to the right.

Standard Normal Table

You can refer to the standard normal table when finding probabilities of normally distributed random variables.

▼ Standard Normal Table:

Example: Let $X$ be normally distributed with mean $3$ and standard deviation $2.$ Find the following:

$P(X < 4)$
$P(X > 2)?$
$P(X < 0)$

Solution: For each case, transform $X$ into a standard normal random variable by subtracting the mean and dividing by the standard deviation. Let $Z$ be a standard normal random variable.

First, standardize the random variable. $\begin{align} P(X < 4) & = P\left(\frac{X-3}{2} < 0.5\right) \\ & = P(Z < 0.5) \end{align}$ In the table, look for the $0.5$ and the $.00$ column, since added together this is $0.50.$ The number found there is $0.69146.$ Therefore, $P(X < 4) \approx 0.69146$
Standardize the normal variable. $\begin{align} P(X > 2) & = P\left(\frac{X-3}{2} > -0.5\right) \\ & = P(Z > -0.5) \end{align}$ By symmetry of the distribution, $P(Z > -0.5) = P(Z \leq 0.5).$ Since $Z$ is continuous, this is the same as $P(Z < 0.5).$ In part $1,$ we found $P(Z < 0.5) \approx 0.69146.$ So, $P(X > 2) \approx 0.69146.$
Standardize the normal variable. $\begin{align} P(X < 0) & = P\left(\frac{X-3}{2} < -1.5\right) \\ & = P(Z < -1.5) \end{align}$ The standard normal table only shows the probability that $Z$ is less than a positive value, so we need to rearrange the inequality.

By symmetry, $P(Z < -1.5) = P(Z > 1.5).$ Using the fact that $P(A) = 1-P(A^C),$ $P(Z > 1.5) = 1 - P(Z \leq 1.5)$ Since $Z$ is continuous, $P(Z \leq 1.5) = P(Z < 1.5).$ This probability can be found in row $1.5$ and column $.00,$ since $1.5+.00 = 1.50.$ The number there is $0.93319.$ Therefore, $P(X < 0) \approx 1-0.93319 = 0.06681$

1. Let $Z$ be a standard normal random variable. Find $P(Z > 0).$

Unanswered

Solution: By symmetry of the standard normal, $P(Z > 0) = 1 - P(Z < 0) = 0.5.$

2. Let $X$ have the Normal $(-2,4)$ distribution. Find $P(X > 0).$

Unanswered

Solution: First standardize $X,$ then use the table for the standard normal distribution. $\begin{align} P(X > 0) & = P(X + 2 > 2) \\ & = P\left(\frac{X+2}{2} > 1\right) \\ & = P\left(Z > 1\right) \end{align}$ where $Z$ is a standard normal random variable. To use the table, taking the complement of $\{Z > 1\}.$ $\begin{align} P(Z > 1) & = 1 - P(Z \leq 1) \\ & = 1 - 0.84134 \\ & \approx 0.16 \end{align}$

3. Let $X$ and $Y$ be normally distributed with mean $0.$ If $Var(X) > Var(Y),$ which is true?

Unanswered

Solution: Let $X$ have variance $\sigma^2$ and $Y$ have variance $\tau^2$ where $\sigma^2 < \tau^2.$ The pdf of $X$ is $f_X(x) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{x^2}{2\sigma^2}}$ and the pdf of $Y$ is $f_Y(y) = \frac{1}{\sqrt{2\pi\tau^2}}e^{-\frac{y}{2\tau^2}}.$ Since $X$ and $Y$ are independent, their joint pdf is $f(x,y)=\frac{1}{2\pi\sigma\tau}e^{-\frac{(x-\mu)^2}{2\sigma^2}-\frac{(y-\mu)^2}{2\tau^2}}.$ Using the joint pdf, we have $P(X < Y) = \int_{-\infty}^\infty \int_y^\infty \frac{1}{2\pi\sigma\tau}e^{-\frac{x^2}{2\sigma^2}-\frac{y^2}{2\tau^2}} dx dy$ Substituting $-x$ for $x$ and $-y$ for $y$ we get $\begin{align} \int_{-\infty}^\infty \int_y^\infty \frac{1}{2\pi\sigma\tau}e^{-\frac{x^2}{2\sigma^2}-\frac{y^2}{2\tau^2}} dx dy & = \int_{-\infty}^\infty \int_{-\infty}^y \frac{1}{2\pi\sigma\tau}e^{-\frac{x^2}{2\sigma^2}-\frac{y^2}{2\tau^2}} dx dy \\ & = P(Y < X) \end{align}$ To see why the bounds on the integral are what they are, notice that the condition for the region we are integrating over changes from $x < y$ to $-x < -y,$ or $x > y.$

So, $P(X < Y) = P(Y < X).$ Since $X$ and $Y$ are continuous, $P(X = Y) = 0,$ so $P(X > Y) = 0.5.$

4. Let $X$ be normally distributed with mean $1$ and variance $2.$ What is $P(0 < X < 2)?$

Unanswered

Solution: First, normalize $X.$ $\begin{align} P(0 < X < 2) & = P\left(\frac{-1}{\sqrt{2}} < \frac{X-1}{\sqrt{2}} < \frac{1}{\sqrt{2}}\right) \\ & = P\left(\frac{-1}{\sqrt{2}} < Z < \frac{1}{\sqrt{2}}\right) \end{align}$ where $Z$ is a standard normal distribution. $\begin{align} P\left(\frac{-1}{\sqrt{2}} < Z < \frac{1}{\sqrt{2}}\right) & = P\left(Z < \frac{1}{\sqrt{2}}\right) - P\left(Z \leq \frac{-1}{\sqrt{2}}\right) \\ & = P\left(Z < \frac{1}{\sqrt{2}}\right) - \left(1 - P\left(Z \leq \frac{1}{\sqrt{2}}\right)\right) \\ & = 2P\left(Z < \frac{1}{\sqrt{2}}\right) - 1 \\ & \approx 2P(Z < 0.71) - 1 \end{align}$ Using the standard normal distribution table, we get $\begin{align} 2P(Z < 0.71) - 1 \approx 2 \cdot 0.76115 - 1 & \approx 0.52 \end{align}$