Transformation technique

for continuous random variables only
Suppose $X \sim f_{X} (x)$ where $f_{X} (x)$ is the pdf of $X$

Let $Y = h (X)$

Bijective transformation

If $h (x)$ is differentiable at all values of $x$ and is either a decreasing/increasing function such that there is a unique inverse function $X = h^{- 1} (Y)$ , then the pdf of $Y$ is

f_{y} (y) = f_{X} (h^{- 1} (y)) \cdot | \frac{\partial h^{- 1} (y)}{\partial y} |

if $\frac{\partial h (x)}{\partial x} = 0, f_{Y} (y) = 0$

Non-bijective transformation

If the transformation is not one-to-one, for example $Y = X^{2}$ , we can divide the domain of the random variable $X$ into multiple non-overlapping regions. Then we find the pdf of $Y$ in each of the regions

f_{Y} (y) = f_{X} (h_{1}^{- 1} (y)) | \frac{d h_{1}^{- 1} (y)}{d y} | + f_{x} (h_{2}^{- 1} (y)) | \frac{| d h_{2}^{- 1} (y)}{d y} |

Multivariable (joint) distributions

Suppose $X_{1}, \dots, X_{n}$ is a set of continuous random variables with joint pdf $f_{X_{1}, \dots, X_{n}} (x_{1}, \dots, x_{n})$ and let $Y_{1} = h_{1} (X_{1}, \dots, X_{n}),$ $Y_{2} = h_{2} (X_{1}, \dots, X_{n}), \dots$ $Y_{n} = h_{n} (X_{1}, \dots, X_{n})$ be another set of random variables.

If the $h_{i}$ functions are differentiable with respect to each of $X_{1}, \dots, X_{n}$ and and are one-to-one correspondent within the range of $X_{1}, \dots, X_{n}$ for which $f_{X_{1}, \dots, X_{n}} (x_{1}, \dots, x_{n}) \neq 0$ , then the joint pdf of $Y_{1}, \dots, Y_{n}$ is given by

f_{Y_{1}, . . ., Y_{n}} (y_{1}, . . ., y_{n}) = f_{X_{1}, . . ., X_{n}} (g_{1} (y_{1}, . . ., y_{n}), . . ., g_{n} (y_{1}, . . ., y_{n})) \cdot | J |

where the $g_{i}$ functions are the inverse transformations

If we have less transformations ( $Y$ ) than random variables $(X)$ , we create new pointless transformations to ensure the Jacobian is a square matrix.
Then we can remove the leftover transformations from the pdf by integrating its marginal.

f_{Y_{1}, \dots, Y_{r}} (y_{1}, \dots, y_{r}) = \int \int \int f_{Y_{1}, \dots, Y_{r}, \dots Y_{n}} (y_{1}, \dots, y_{r}, \dots, y_{n}) d y_{n - 2} d y_{n - 1} d y_{n}

Examples

$X_{1}, X_{2} \sim E x p o n e n t i a l$

\begin{aligned} Y_{1} & = X_{1} + X_{2}, & X_{1} & = Y_{2} \\ Y_{2} & = X_{1}, & X_{2} & = Y_{1} - Y_{2} \end{aligned}

As both random variables are independent:

\begin{aligned} f_{X_{1}, X_{2}} (x_{1}, x_{2}) & = \frac{1}{β} e^{- x_{1} / β} \cdot \frac{1}{β} e^{- x_{2} / β} \\ f_{X_{1}, X_{2}} (x_{1}, x_{2}) & = \frac{1}{β^{2}} e^{- (x_{1} + x_{2}) / β} \end{aligned}

Jacobian:

| J | = det [\begin{matrix} \frac{\partial X_{1}}{\partial Y_{1}} & \frac{\partial X_{1}}{\partial Y_{2}} \\ \frac{\partial X_{2}}{\partial Y_{1}} & \frac{\partial X_{2}}{\partial Y_{2}} \end{matrix}] = det [\begin{matrix} 0 & 1 \\ 1 & - 1 \end{matrix}] = | - 1 | = 1

Joint pdf of $Y_{1}, Y_{2} :$

\begin{aligned} f_{Y_{1}, Y_{2}} (y_{1}, y_{2}) & = f_{X_{1}, X_{2}} (Y_{2}, Y_{1} - Y_{2}) \cdot 1 \\ = \frac{1}{β^{2}} e^{- (y_{2} + y_{1} - y_{2}) / β} \\ = \frac{1}{β^{2}} e^{- y_{1} / β} 0 \leq y_{2} \leq y_{1} \end{aligned}

Marginalizing:

\begin{aligned} f_{Y_{1}} (y) & = \int_{0}^{y_{1}} \frac{1}{β^{2}} e^{- y_{1} / β} d y_{2} \\ = \frac{1}{β^{2}} e^{- y_{1} / β} \int_{0}^{y_{1}} 1 d y_{2} \\ f_{Y_{1}} (y) & = \frac{1}{β^{2}} y_{1} e^{- y_{1} / β}, y_{1} \geq 0 \end{aligned}

The transformation $Y_{1}$ follows a Gamma distribution with parameters $α = 2, β = β$

Finding the pdf of a cdf

\begin{aligned} Y & = g (x) = F_{X} (x) & 0 \leq F_{X} (x) \leq 1, 0 \leq y \leq 1 \\ X & = h^{- 1} (Y) & \frac{d x}{d y} = \frac{1}{\frac{d y}{d x}} = \frac{1}{\frac{d F_{X} (x)}{d x}} & = \frac{1}{f_{X} (x)} = \frac{d h^{- 1} (y)}{d y} \\ f_{Y} (y) & = f_{X} (h^{- 1} (y)) \cdot | \frac{d h^{- 1} (y)}{d y} | \\ = 1, Y = F_{X} (x) \sim u n i f [0, 1] \end{aligned}

The cdf for any pdf follows a Uniform Distribution