04 - Joint Distributions

Joint Cumulative Distribution Function

Let $X$ and $Y$ be two random variables then

Joint Cumulative Distribution Function is defined by

$$F_{X,Y}(x,y)=\mathbb{P}(X\le x,Y\le y)$$

Jointly Continuous

Let $X$ and $Y$ be two random variables then

$X$ and $Y$ are jointly continuous if joint cdf can be written as

$$F_{X,Y}(x,y)={\int }_{u=-\infty }^x{\int }_{v=-\infty }^{y}f(u,v)dudv$$

for some $f_{X,Y}$

Joint PDF

Let $X$ and $Y$ be two random variables then

Joint PDF is defined by

$$f_{X,Y}(x,y)=\frac{{\partial }^2}{\partial x\partial y}{F}_{X,Y}(x,y)$$

with probability function

$$\mathbb{P}((X,Y)\in A)={\iint }_A{f}_{X,Y}(x,y)dxdy\text{ for general sets }A\subset {\mathbb{R}}^2$$

Marginal Distribution

Let $X$ and $Y$ be two random variables then

Marginal Distribution of $X$ (similar for $Y$) is defined by

$$f_X(x)={\int }_{y=-\infty }^{\infty }{f}_{X,Y}(x,y)dy$$

Expectation of Two Variables

Let $X$ and $Y$ be two random variables then

$$\mathbb{E}[h(X,Y)]={\int }_{x=-\infty }^{\infty }{\int }_{y=-\infty }^{\infty }h(x,y)f_{X,Y}(x,y)dxdy$$

Independent of Two Variables

Let $X$ and $Y$ be two random variables then

$X$ and $Y$ are independent if

$$F_{X,Y}(x,y)=F_X(x)F_Y(y)\text{ for all }x,y$$

or

$$f_{X,Y}(x,y)=f_X(x)f_Y(y)$$

Convolution of Sum of Independent Continuous Variables

Let $X$ and $Y$ be two independent random variables then

$$f_{X+Y}(u)={\int }_{-\infty }^{\infty }{f}_X(v)f_Y(u-v)dv$$

Joint Density of i.i.d. Standard Normals

Let $Z_1,Z_2,\cdots ,Z_n$ be i.i.d standard normal random variables then

Joint Density Function is defined as

$$\begin{array}{rl}{f}_{\mathbf{Z}}(\mathbf{z})& =\prod _{i=1}^n\frac{1}{\sqrt{2\pi }}\exp \left(-\frac{z_i^2}{2}\right)\\ & =\frac{1}{(2\pi {)}^{\frac{n}{2}}}\exp \left(-\frac{1}{2}{\mathbf{z}}^T\mathbf{z}\right)\end{array}$$

Density of an Affine Transformation of Standard Normals

Let $Z_1,Z_2,\cdots ,Z_n$ be i.i.d standard normal random variables

Define $W_1,\cdots ,W_n$ by

$$\left(\begin{array}{c}{W}_1\\ W_2\\ ⋮\\ W_n\end{array}\right)=A\left(\begin{array}{c}{Z}_1\\ Z_1\\ ⋮\\ Z_n\end{array}\right)+\left(\begin{array}{c}{\mu }_1\\ {\mu }_2\\ ⋮\\ {\mu }_n\end{array}\right)$$

where $A$ is some $n\times n$ matrix (assume invertible)

By Change of Variables (with Jacobian Constant as Linear Transformation) then

$$f_{\mathbf{W}}(\mathbf{w})=\frac{1}{(2\pi {)}^{\frac{n}{2}}|\det A|}\exp \left(-\frac{1}{2}(\mathbf{w}-\mu {)}^T(A{A}^T{)}^{-1}(\mathbf{w}-\mu {)}_{}\right)$$

Covariance Matrix

Let $Z_1,Z_2,\cdots ,Z_n$ be i.i.d standard normal random variables

Define $W_1,\cdots ,W_n$ by

$$\left(\begin{array}{c}{W}_1\\ W_2\\ ⋮\\ W_n\end{array}\right)=A\left(\begin{array}{c}{Z}_1\\ Z_1\\ ⋮\\ Z_n\end{array}\right)+\left(\begin{array}{c}{\mu }_1\\ {\mu }_2\\ ⋮\\ {\mu }_n\end{array}\right)$$

where $A$ is some $n\times n$ matrix (assume invertible)

Covariance Matrix is defined as

$$\Sigma :=A{A}^T$$

so that

$$\text{cov}(W_i,W_j)=(A{A}^T{)}_{ij}={\Sigma }_{ij}$$

Multivariate Normal Distribution

Let $Z_1,Z_2,\cdots ,Z_n$ be i.i.d standard normal random variables

Define $W_1,\cdots ,W_n$ by

$$\left(\begin{array}{c}{W}_1\\ W_2\\ ⋮\\ W_n\end{array}\right)=A\left(\begin{array}{c}{Z}_1\\ Z_1\\ ⋮\\ Z_n\end{array}\right)+\left(\begin{array}{c}{\mu }_1\\ {\mu }_2\\ ⋮\\ {\mu }_n\end{array}\right)$$

where $A$ is some $n\times n$ matrix (assume invertible)

$W_1,\cdots ,W_n$ have multivariate normal distribution with mean $\mu$ and covariance matrix $\Sigma$