Functions of Random Variable
Change of Variable
Theorem Let \(X\) be a r.v. and \(Y=g(X)\) where \(g:\mathbb R\rightarrow\mathbb R\) is a funciton. Then
\[\text{pmf}_Y(y) = \sum_{x: g(x) = y} \text{pmf}_X(x)\]
- \(X\) is continuous and \(g\) is an appropriate transformation,
\[\text{cdf}_Y(y) = \int_{x: g(x)\leq y}\text{pdf}_X(x)dx = P(\{x: g(x) \leq y\})\]
\[\text{pdf}_Y(y) = \frac{d}{dy} \text{cdf}_Y(y)\]
Theorem Let \(F(x) = \text{cdf}_X(x)\), then \(F(X)\sim uniform(0, 1)\)
proof. Let \(Y=F(X)\),
\[\text{cdf}_Y(y) = P(\{x:F(x) < y\}) = P(X \leq x) = F(x) = y\]
Change of Single Variable
Theorem (change of variable) Let \(X\) be continuous r.v. and function \(g\) be differentiable and injective. Then
\[\text{pdf}_Y(y) = \text{pdf}_X(g^{-1}(y))|\frac{d}{dy}g^{-1}(y)|\]
proof. wlog assume \(g\) is increasing (as a appropriate transformation function), then
\begin{align}
\text{pdf}Y(y) &= \frac{d}{dy}\text{cdf}_Y(y)\
&= \frac{d}{dy}\int{g_X(x)dx\
&= \frac{d}{dy} \text{cdf}_X(g^{-1}(y)) \
&= \text{pdf}_X(g}(y)} \text{pdf{-1}(y))|\frac{d}{dy}g(y)|
\end{align}
Change of Variables for Multivariate Functions
Theorem For discrete random variables \(\mathbf X = (X_1,...,X_n)\), Let \(\mathbf G:\mathbb R^n\rightarrow\mathbb R^m\) be the transformation s.t. \(\mathbf Y = (Y_1,...,Y_m), \mathbf Y = \mathbf G(\mathbf X), Y_i = g_i(X_1, ..., X_n)\). Then
\[\text{pmf}_{\mathbf Y} = \sum_{\mathbf x: \mathbf G(\mathbf x) = \mathbf y}\text{pmf}_{\mathbf X}(\mathbf x)\]
random variables \(X_1,...,X_n\) are said to be independent and identically distributed (iid.) if \(X_i\)'s are independent and have the same distribution.
Example
the sum of independent Bernoulli trails follows binomial distribution.
proof. Let \(X_i \sim \text{Bern.}(p)\). \(Y_n = \sum^n X_i\). We will prove by induction.
Obviously \(Y_1 = X_1 \sim \text{Bern.}(p)\equiv \text{binomial}(1, p)\)
Assume \(Y_k \sim \text{binomial}(k, p)\). Then
\[\begin{align*}
P(Y_{k+1} = 0) &= P(Y_k=0, X_{k+1} - 0) \\
&= P(Y_k = 0)P(X_{k+1}=0)\\
&= {k\choose 0}(1-p)^k (1-p) \\
&= {k+1\choose 0}(1-p)^{k+1}\\
P(Y_{k+1}=j) &= P((Y_k = j, X_{k+1} = 0)\cup (Y_k = j-1, X_{k+1} = 1))\\
&= P(Y_k = j)P(X_{k+1} = 0) + P(Y_k = j-1)P(X_{k+1} = 1)\\
&= {k\choose j}p^j(1-p)^{k-j}(1-p) + {k\choose j-1}p^{j-1}(1-p)^{k-(j-1)}p\\
&= {k+1\choose j}p^j(1-p)^{k+1-j}\\
Y_{k+1}&\sim \text{binomial} (k+1,p)
\end{align*}\]
Theorem
If \(X,Y\) are independent continuous r.v. then
\[\text{pdf}_{X+Y}(z) = \int \text{pdf}_X(x)\text{pdf}_Y(z-x)dx\]
proof. Let \(Z=X+Y\)
\[\begin{align*}
\text{cdf}_Z(z) &= P(X+Y \leq z)\\
&= P(X \leq x, Y \leq z-x)\\
&=P(X \leq x)P(Y \leq z-x)\\
&= \int_{-\infty}^{\infty} \int_{-\infty}^{z-x}\text{pdf}_X(x)\text{pdf}_Y(y)dydx\\
&= \int_{-\infty}^{\infty} \text{pdf}_X(x) \text{cdf}_Y(z-x)dx\\
\text{pdf}_Z(z) &= \frac{d}{dz} \int_{-\infty}^{\infty} \text{pdf}_X(x) \text{cdf}_Y(z-x)dx\\
&= \int_{-\infty}^{\infty} \text{pdf}_X(x) \text{pdf}_Y(z-x)dx\\
\end{align*}\]
Example
For \(X_1,...,X_n\) iid, \(Y_n = \max(X_1,...,X_n), Y_1 = \min(X_1,...,X_n)\).
\[\begin{align*}
\text{cdf}_{Y_n}(y) &= P(\max(X_1,...,X_n) \leq y) \\
&= P(X_1 \leq y,...,X_n\leq y)\\
&= \prod^n P(X_i \leq y) \\
&= \text{cdf}_X(y)^n\\
\text{cdf}_{Y_1}(y) &= 1 - P(Y_1 > y)\\
&= 1 - P(\min(X_1,...,X_n) > y) \\
&= 1 - P(X_1 > y,...,X_n > y)\\
&= 1 - \prod^n P(X_i > y) \\
&= 1 - (1-\text{cdf}_X(y))^n\\
\text{cdf}_{Y_1,Y_n}(y_1,y_2)&= P(Y_1\leq y_1, Y_2\leq y2)\\
&= P(y_2\leq y_2) - P(Y_1 > y_1, Y_2\leq y_2)\\
&= \text{cdf}_X(y_2)^n - \prod^n P(y_1 < X_i \leq y_2)\\
&= \text{cdf}_X(y_2)^n - (\text{cdf}_X(y_2) - \text{cdf}_X(y_1))^n
\end{align*}\]
Theorem (change of variables)
For random variables \(\mathbf X = (X_1,...,X_n)\), Let \(\mathbf G:\mathbb R^n\rightarrow\mathbb R^m\) be the transformation s.t. \(\mathbf Y = (Y_1,...,Y_m), \mathbf Y = \mathbf G(\mathbf X), Y_i = g_i(X_1, ..., X_n)\). IF \(\mathbf G\) is injective and differentiable, then
\[\text{pdf}_{\mathbf Y}(\mathbf y) = \text{pdf}_{\mathbf X}(\mathbf G^{-1}(\mathbf y))|\det(\frac{D}{D\mathbf y}\mathbf G^{-1}(\mathbf y))|\]