Probability Spaces

Week 1

Reading

FT-1 | Probability Space

The purpose of probability is to obtain answers about real-life phenomena which do not have a predetermined outcome. We need a mathematical model to describe this so that we can provide answers to these questions. To this end, we define a probability space.

This space is defined on the sample space of all possible outcomes of a random experiment, denoted as $\mathrm{\Omega }$. Each element (outcome) of $\mathrm{\Omega }$ is denoted by $𝜔$, and a collection of outcomes is called an event, aka a subset of $\mathrm{\Omega }$. Probability requires us to understand the size of an event. Not all subsets of $\mathrm{\Omega }$ can be measured, so we will define a collection of subsets on which a measure can be defined, the $𝜎$-algebra.

A set is described in braces like so:

$\left\{𝑥|𝑥\text{satisfy property}\mathrm{𝒫︀}\right\}$

The set which does not contain an element is the empty set $\varnothing$.

To say that an element belongs to a set is that $𝜔\in 𝐴$, and for it not to belong is $𝜔\notin 𝐴$.

A union of sets $𝐴\cup 𝐵$ has elements that are in either of them.

An intersection of sets $𝐴\cap 𝐵$ has elements that are in both of them.

The complement ${𝐴}^{𝑐}$ of set $𝐴$ is the set of elements in $\mathrm{\Omega }$ but not in $𝐴$

The difference of two sets $𝐴\setminus 𝐵$ is the set of elements in $𝐴$ but not $𝐵$

The symmetric difference of two sets is $𝐴\mathrm{\Delta }𝐵=(𝐴\setminus 𝐵)\cup (𝐵\setminus 𝐴)$, which is the reverse of $𝐴\cap 𝐵$ if $𝐴$ and $𝐵$ share elements. In terms of a Venn diagram, it would be the left and right circle parts without the middle part.

If $𝐴\cap 𝐵=\varnothing$, then these sets are considered disjoint or mutually exclusive.

There is a distributive law (DeMorgan’s law) for intersection over union:

$𝐴\cap (𝐵\cup 𝐶)=(𝐴\cap 𝐵)\cup (𝐴\cap 𝐶)$

and the reverse for unions.

Let’s consider all possible subsets of $\mathrm{\Omega }$ as $\mathrm{𝒫︀}(\mathrm{\Omega })$. An algebra on $\mathrm{\Omega }$ is a collection of sets in $\mathrm{𝒫︀}(\mathrm{\Omega })$ which is closed under complementarity and finite union.

There are three basic properties of the algebra $\mathrm{𝒜︀}$:

• $\mathrm{\Omega }\in \mathrm{𝒜︀}$
• If $𝐴\in \mathrm{𝒜︀}$, then ${𝐴}^{𝑐}\in \mathrm{𝒜︀}$
• If $𝐴,𝐵\in \mathrm{𝒜︀}$, then $𝐴\cup 𝐵\in \mathrm{𝒜︀}$

These properties naturally extend themselves to other properties, like the empty set being in the algebra, intersections being in the algebra, and the union of all different sets being in the algebra.

A $𝜎$-algebra is a kind of algebra which is closed under countable unions. Regular algebras can have uncountable unions. These kinds of unions arise in stochastic processes. In order to use $𝜎$-algebras when describing stochastic processes, we will define filtrations that increase $𝜎$-algebras over time. When we try to prove properties of $𝜎$-algebras, we will instead prove properties about their generators, and then extend them to the $𝜎$-algebra.

Let’s consider a collection $\mathrm{𝒞︀}$ of sets in $\mathrm{\Omega }$. The $𝜎$-algebra of $\mathrm{𝒞︀}$ defined as $𝜎(\mathrm{𝒞︀})$ has the properties that it contains all sets in $\mathrm{𝒞︀}$ and it is the smallest $𝜎$-algebra which is a subset of any other $𝜎$-algebra which contains $\mathrm{𝒞︀}$.

Here are some facts about $𝜎$-algebras:

$\mathrm{𝒫︀}(\mathrm{\Omega })$ is the largest possible $𝜎$-algebra on $\mathrm{\Omega }$.

If $\mathrm{𝒞︀}$ is already a $𝜎$-algebra, then the generated $𝜎$-algebra is the same.

If $𝐶$ is $\varnothing$ or $\mathrm{\Omega }$, $𝜎(\mathrm{𝒞︀})$ is $\left\{\varnothing ,\mathrm{\Omega }\right\}$, the trivial $𝜎$-algebra.

Any collection of sets which has $𝐶$ has subset has the same $𝜎$-algebra as $𝐶$.

There are a lot of elements of $𝜎$-algebras, so we typically consider them in the context of generator $\mathrm{𝒞︀}$.

A measurable space is a pair $(\mathrm{\Omega },\mathrm{ℱ︀})$ where $\mathrm{\Omega }$ is the sample space and $\mathrm{ℱ︀}$ is a $𝜎$-algebra on $\mathrm{\Omega }$.

For two $𝜎$-algebras ${\mathrm{ℱ︀}}_1$ and ${\mathrm{ℱ︀}}_2$, ${\mathrm{ℱ︀}}_1\cap {\mathrm{ℱ︀}}_2$ is a $𝜎$-algebra (and in fact this can be generalized to any countable intersection of $𝜎$-algebras, but ${\mathrm{ℱ︀}}_1\cup {\mathrm{ℱ︀}}_2$ is not a $𝜎$-algebra. You would have to define $𝜎({\mathrm{ℱ︀}}_1\cup {\mathrm{ℱ︀}}_2)$, better denoted as ${\mathrm{ℱ︀}}_1\vee {\mathrm{ℱ︀}}_2$.

To picture the Borel $𝜎$-algebra, we must first envision our $\mathrm{\Omega }$ as a topological space where geometry is defined. We will specifically consider $\mathrm{\Omega }=ℝ$, where our sample space is the real number line. The Borel $𝜎$-algebra $\mathrm{ℬ︀}$ is generated by the class of open subsets of $\mathrm{\Omega }$. Think of this as every subdivision of the number line. Almost every subset of $ℝ$ is in $\mathrm{ℬ︀}$, except for some very exotic sets.

As an example: if you partition $\mathrm{\Omega }$ i.e. you split it into disjoint sets, the different unions of all those sets is the same as the $𝜎$-algebra generated by all the disjoint sets. This can be proven by the fact that the complement of each collection of sets is present as a union of different sets, since they encompass the whole sample space.