Random Variables and CDFs

Week 3

Reading - Random Variables Generalities (FT 3)

Definition

We can consider a mapping between all of the sets in the $𝜎$-algebra of one measurable space $(\mathrm{\Omega },\mathrm{ℱ︀})$ to another sample space ${\mathrm{\Omega }}_1$, where the mapping is measurable each set has a mapping. A random variable is a measurable function $𝑋:\mathrm{\Omega }\to ℝ$ where the target sample space is the Borel set. Random variables are capital letters and their values are lowercase letters.

The sum, product, or composition of measurable functions are measurable functions, so those are still random variables.

We consider the preimage of $𝑋$ to be ${𝑋}^{-1}(𝐴)$, which is not precisely the inverse function. It’s the set in the $𝜎$-algebra in the input sample space which generated the set $𝐴$.

Distribution

The distribution of a random variable is itself a random variable, denoted by ${\mathbb{P}}_{𝑋}=\mathbb{P}\circ {𝑋}^{-1}(ℝ)$. This variable is defined on the Borel set as a probability on the measurable space. We can consider the probability space of the distribution to be $(ℝ,𝔹(ℝ),\mathbb{P}𝑋)$ i.e. the sample space is all real numbers, the function mapping is the Borel subset (every interval and combination of such), and the distribution is the probability measure.

If we have a sequence of disjoint sets in the Borel subset of $ℝ$, then the distribution of $𝑋$ ($\mathbb{P}(𝑋)$) on the unions of those sequences is the same as the sum of the probability measure on each individual set.

We will consider the notation $\left\{𝑋\in 𝐴\right\}$ to mean that there’s an outcome $𝜔$ in sample space $\mathrm{\Omega }$ $𝑋$ which maps to the set $𝐴$ in $\mathrm{ℬ︀}(ℝ)$

Cumulative Distribution Function

Formally, the cdf of a random variable is $𝐹(𝑥)=\mathbb{P}(\left\{𝜔:𝑋(𝜔)\le 𝑥\right\})$ or the probability measure of the outcomes where the mapping to the real numbers is less than a specified number x.

This function has the properties that it is increasing, right continuous, and it approaches 0 and 1 from left and right, respectively.

The infinimum $inf$ is the largest element smaller than every element of a set (aka it’s the element just to the left of the set in the order) and the supremum $sup$ is the element to the right.

If we consider the variable.

For every cdf that exists, there exists a random variable that has that cdf.

A Borel function is a measurable function that maps from $ℝ$ into $ℝ$.

Lecture Notes

CDF

$𝐹:ℝ\to [0,1]$ with $𝐹(𝑥)≔{𝑃}_{𝑋}((-\infty ,𝑥])={𝑃}_{𝑋}(𝐵)$, where $𝐵=(-\infty ,𝑥]$

These are equivalent ways of writing a cdf. This counts the accumulated probability less than or equal to x.

More interestingly, the cdf has these properties: increasing

The increasing property is fairly intuitive. Rigorously, you can show that for $𝑦\ge 𝑥$, $(-\infty ,𝑥]\subseteq (-\infty ,𝑦]$. The interval from x to y must be 0 or greater, so then the cdf to y must be greater than the cdf to x.

You can also show that it’s right continuous, proof is beyond the scope, involves monotone class

${lim}_{ℎ\searrow ︎0}𝐹(𝑥+ℎ)=𝐹(𝑥)$ for any x in the domain.

Let’s examine the first quadrant of an equation. We have $𝑥$ . If we imagine the difference between that and $𝑥+ℎ$, in a continuous function this is well-defined. But we might have a discontinuous function. This cannot be a cumulative distribution function. This is a complicated statement.

${lim}_{𝑥\to -\infty }𝐹(𝑥)=0,{lim}_{𝑥\to \infty }𝐹(𝑥)=1$

We can prove this by showing that

${lim}_{𝑥\to -\infty }𝐹(𝑥)=0={lim}_{𝑥\to -\infty }{𝑃}_{𝑋}((-\infty ,𝑥])={𝑃}_{𝑋}(\varnothing )=0$

Why is this empty set? Because if you have the interval $(-\infty ,\infty ]$, left open interval says I don’t have it, right says I do, because they conflict then there must be an empty set.

${lim}_{𝑥\to \infty }𝐹(𝑥)=1={𝑃}_{𝑋}((-\infty ,\infty ])={𝑃}_{𝑋}(ℝ)=1$

This just becomes the probability measure of the sample space which is definitionally 1.

Most of the focus in this course will be calculation and not proof. But if you understand the proof, it will guide you in calculations.

Lemma 3.11: If we have $𝐹(𝑋)$ as the cdf, then

$\mathbb{P}(𝑋\ge 𝑥)=1-\mathbb{P}(𝑋\le 𝑥)$

This is known as the survival function. If you survive over sixty years, then you get your retirement money. This assumes that $\mathbb{P}(𝑋=𝑥)=0$ which is true if $𝑋$ is a continuous random variable.

Proof is $\left\{𝑋>𝑥\right\}\cup \left\{𝑋\le 𝑥\right\}=ℝ=\mathrm{\Omega }$

Important identity:

$\mathbb{P}(𝑥<𝑋\le 𝑦)=𝐹(𝑦)-𝐹(𝑥)$

This proof comes from the set inequality: $\left\{𝑋\le 𝑥\right\}\cup \left\{𝑥<𝑋\le 𝑦\right\}=\left\{𝑋\le 𝑦\right\}$

And then if we take the probability measure of all of these then we see that the right side is the cdf of y, left side is cdf of x disjoint with the other part so you can add them by earlier definition

Also known

$\mathbb{P}(𝑋=𝑥)=𝐹(𝑥)-𝐹(𝑥-)$ this is us taking the left limit where

$𝐹(𝑥-)≔{lim}_{𝑦\nearrow ︎𝑥}𝐹(𝑦)={lim}_{ℎ\searrow ︎0}𝐹(𝑥-ℎ)$

consequences of this? If this $𝐹(𝑥)$ is left-continuous, it means that the limit is equal, which means this whole thing is equal to 0. However, it’s not left-continuous, then $\mathbb{P}(𝑋=𝑥)>0$ strictly larger than 0. This looks like when the dot is on the right and the open circle is on the left. This is allowed, and it happens when $𝑋$ is a discrete random variable. Binomial, Poisson, Geometric random variables are all discrete.

Numerical

Roll a die, probability that number is smaller than five. (strictly less than on an exam)

$\mathbb{P}(𝑋<5)=\mathbb{P}(𝑋\le 4)=𝐹(4)=\mathbb{P}(\left\{𝑋=1\right\}\cup \left\{𝑋=2\right\}\cup \left\{𝑋=3\right\}\cup \left\{𝑋=4\right\})$

$=\mathbb{P}(𝑋=1)+\mathbb{P}(𝑋=2)+\mathbb{P}(𝑋=3)+\mathbb{P}(𝑋=4)$

$=\frac{1}{6}+\frac{1}{6}+\frac{1}{6}+\frac{1}{6}$

$=\frac{2}{3}$

alternatively, I could do the same with 5 and 6 and use the 1 - property. Two random variables are independent if

$\mathbb{P}(𝑋\in 𝐴,𝑌\in 𝐵)=\mathbb{P}(𝑋\in 𝐴)\times \mathbb{P}(𝑌\in 𝐵)$

or put another way

$=\mathbb{P}(\left\{𝑋\in 𝐴\right\}\cap \left\{𝑌\in 𝐵\right\})$

Intuitively, this means that the outcome of $𝑋$ does not affect the outcome of $𝑌$.

For any $𝑥,𝑦\in ℝ$, where $𝑥,𝑦$ are independent

$\mathbb{P}(𝑋\le 𝑥,𝑌\le 𝑦)=\mathbb{P}(𝑋\le 𝑥)\times \mathbb{P}(𝑌\le 𝑦)$

are mutually independent if

$\mathbb{P}\left({\bigcap }_{𝑖=1}^{𝑛}\left\{{𝑥}_{𝑖}\in {𝐵}_{𝑖}\right\}\right)={\prod }_{𝑖=1}^{𝑛}\mathbb{P}({𝑋}_{𝑖}\in {𝐵}_{𝑖})$

They are also pairwise independent in this case.

For an arbitrary family of random variables where we have some index set $𝐼$ for ${𝑋}_{𝑖}$, they are mutually independent if every finite subfamily is independent. That means any finite subset of the indices will cause this to be the case. This is not very useful as a way to prove independence because there are so many cases to prove, this is just a definition.

Let’s say we roll a fair die twice and we have $𝑋$ represent the maximum of both rolls. The first question we’re interested in calculating the image of $\mathrm{\Omega }$ through $𝑋$ and the distribution of $𝑋$.

$\mathrm{\Omega }=\left\{𝜔=({𝑗}_1,{𝑗}_2),{𝑗}_1,{𝑗}_2\in \left\{1,2,3,4,5,6\right\}\right\}$

total of 36 elements

the probability of observing any particular outcome is $\frac{1}{36}$ because it’s fair. This $𝑋$ is clearly a mapping from the sample space to the set of six numbers. If we calculate probabilities:

$\mathbb{P}(𝑋=1)={\mathbb{P}}_{𝑋}(𝜔=(1,1))=\frac{1}{36}$

$\mathbb{P}(𝑋=2)={\mathbb{P}}_{𝑋}(𝜔=(1,2))+{\mathbb{P}}_{𝑋}(𝜔=(2,1))+{\mathbb{P}}_{𝑋}(𝜔=(2,2))=\frac{3}{36}$

$\mathbb{P}(𝑋=3)=\mathbb{P}\left({\bigcup }_{𝑘=1}^3\left\{𝜔=(3,𝑘\right\}\cup {\bigcup }_{𝑘=1}^2\left\{𝜔=(𝑘,3)\right\}\right)=\frac{5}{36}$

This is 3 then 2 because you don’t want to double count the same number.

$\mathbb{P}(𝑋=4)=\mathbb{P}\left({\bigcup }_{𝑘=1}^4\left\{𝜔=(4,𝑘\right\}\cup {\bigcup }_{𝑘=1}^3\left\{𝜔=(𝑘,4)\right\}\right)=\frac{7}{36}$

$\mathbb{P}(𝑋=5)=\mathbb{P}\left({\bigcup }_{𝑘=1}^5\left\{𝜔=(5,𝑘\right\}\cup {\bigcup }_{𝑘=1}^4\left\{𝜔=(𝑘,5)\right\}\right)=\frac{9}{36}$

$\mathbb{P}(𝑋=6)=\mathbb{P}\left({\bigcup }_{𝑘=1}^6\left\{𝜔=(6,𝑘\right\}\cup {\bigcup }_{𝑘=1}^5\left\{𝜔=(𝑘,6)\right\}\right)=\frac{11}{36}$

Now let’s get the cdf $𝐹(𝑋)$. Let’s consider each instance of $𝐹(𝑏)=\mathbb{P}(𝑋\le 𝑏)$

Obviously, this has to be 0 for less than 1 because that’s not an outcome. For the rest of them, you add up the probabilities (because they’re disjoint).

Another example.

You are given

$\mathbb{P}(0\le 𝑋\le 1)=\frac{8}{12}$

$\mathbb{P}(0\le 𝑋\le 2)=\frac{7}{12}$

$\mathbb{P}(0\le 𝑋\le 3)=\frac{10}{12}$

$\mathbb{P}(𝑋=3)=\mathbb{P}(𝑋\ge 4)$

We want to know the probability for up to 3.

We can solve for $\mathbb{P}(𝑋=0)=\frac{3}{12}$

We also want to show this inequality $𝑋(𝜔)\le 𝑌(𝜔),\forall 𝜔\in \mathrm{\Omega }$

You want to show that ${𝐹}_{𝑋}(𝑎)\ge {𝐹}_{𝑌}(𝑎),𝑎\in ℝ$

Hint: $𝐹(𝑋(𝑎)=\mathbb{P}(𝑋\le 𝑎)?\mathbb{P}(𝑦\le 𝑎)={𝐹}_{𝑌}(𝑎)$

More interesting example 3.4

cdf

you have to find the conditions on a, b, c, d, that $𝐹$ satisfies the properties of a cdf.

We have the property

${lim}_{𝑥\to \infty }𝐹(𝑥)=1=𝑑$

Because that’s if $𝑡\ge 3$.

a gives you the same solution of 0.