Random Vectors and Independence

Week 10

Lecture Notes

Random Vectors

Beginning chapter 7, vectors of random variables. We are concerned with a random vector

$𝑋=({𝑋}_1,\dots ,{𝑋}_{𝑁})$

The cdf of this vector is

${𝐹}_{𝑋}({𝑥}_1,\dots ,{𝑥}_{𝑁})=\mathbb{P}({𝑋}_1\le {𝑥}_1,\dots {𝑋}_{𝑁}\le {𝑥}_{𝑛})$

We can also say that if we have a random vector (X, Y) then

If you compute joint cdf, we can get marginal distribution (distribution for a subset, maybe one element) very easily

${𝐹}_{𝑋}(𝑥)={lim}_{𝑦\to \infty }{𝐹}_{(𝑋,𝑌)}(𝑥,𝑦)$

and equivalently

$\mathbb{P}(𝑋\le 𝑥)={lim}_{𝑦\to \infty }\mathbb{P}(𝑋\le 𝑥,𝑌\le 𝑦)$

Remark 7.7 says components of vectors are independent

These results hold for every random vector.

Discrete Random Vectors

Now we will talk about discrete random vectors.

The formal definition 7.8 says that the rv is discrete when the image set $𝑋(\mathrm{\Omega })$ is a finite or countable subset of ${ℝ}^{𝑛}$, the product space between n different real number lines.

The pmf for a discrete random vector is given by the basic formula

$\mathbb{P}({𝑋}_1={𝑥}_1,\dots {𝑋}_{𝑁}={𝑥}_{𝑁})$

The density dictates that the sum must be 1.

${\sum }_{𝑥\in 𝑋(\mathrm{\Omega })}\mathbb{P}(𝑋=𝑥)=1$

Then the cdf is

${𝐹}_{𝑋}({𝑥}_1,\dots ,{𝑥}_{𝑛})={\sum }_{{𝑢}_1\le {𝑥}_1}\dots {\sum }_{{𝑢}_{𝑁}\le {𝑥}_{𝑁}}\mathbb{P}({𝑋}_1={𝑢}_1,\dots ,{𝑋}_{𝑁}={𝑢}_{𝑁})$

In order to intuitively understand why the marginal distribution calculation works for joint distribution, imagine a table of values, which has the probability for each event in the sample space. The rows are of the first random variable’s sample space, and the columns are for the second. Then, in order to calculate the probability of an event happening for the second random variable, you need to total all the probabilities from each row in the table.

Independence

Example 7.4

We have (X, Y) random vector. where

$𝑋(\mathrm{\Omega })=𝑌(\mathrm{\Omega })=\left\{0,1\right\}$

and

$\mathbb{P}(𝑋=0,𝑌=0)=(1-𝑝{)}^2$

$\mathbb{P}(𝑋=1,𝑌=1)={𝑝}^2$

$\mathbb{P}(𝑋=0,𝑌=1)=\mathbb{P}(𝑋=1,𝑌=0)=𝑝(1-𝑝)$

How do we prove independence?

We need to prove

$\mathbb{P}(𝐴\cap 𝐵)=\mathbb{P}(𝐴)\mathbb{P}(𝐵)$

or for joint distributions,

$\mathbb{P}(𝑋=𝑥,𝑌=𝑦)=\mathbb{P}(𝑋=𝑥)\mathbb{P}(𝑌=𝑦)$

We can use the marginal distribution calculation for this.

$\mathbb{P}(𝑋=0)=\mathbb{P}(𝑋=0,𝑌=0)+\mathbb{P}(𝑋=0,𝑌=1)=(1-𝑝{)}^2+𝑝(1-𝑝)=1-𝑝$

$\mathbb{P}(𝑋=1)={𝑝}^2+𝑝(1-𝑝)=𝑝$

$\mathbb{P}(𝑌=0)=1-𝑝$

$\mathbb{P}(𝑌=1)=𝑝$

And then now you can verify for each combination of different X and Y that the independence equality holds. Left as exercise for reader 😅

Continuous Random Vectors

We can consider any function $𝑓:{ℝ}^{𝑛}\to ℝ$ a density if it’s positive and

${\int }_{{ℝ}^{𝑛}}𝑓(𝑥)𝑑𝑥={\int }_{{ℝ}_1}\dots {\int }_{{ℝ}_{𝑁}}𝑓({𝑥}_1,\dots ,{𝑥}_{𝑛})𝑑{𝑥}_1,\dots 𝑑{𝑥}_{𝑛}=1$

We can say that the random vector has this density given by f if

$\mathbb{P}(({𝑋}_1,\dots {𝑋}_{𝑛})\in 𝐷)={\int }_{𝐷}𝑓({𝑥}_1,\dots ,{𝑥}_{𝑛})𝑑{𝑥}_1\dots 𝑑{𝑥}_{𝑛}$

Where D is the product space. And in fact, for the cdf, this D is the Borel set

For the marginal density, instead of a summation of all the others, you actually do the integral of all the others

${𝑓}_{{𝑥}_{𝑖}}({𝑥}_{𝑖})={\int }_{{ℝ}^{𝑛-1}}𝑓({𝑥}_1,\dots ,{𝑥}_{𝑛})𝑑{𝑥}_1\dots 𝑑{𝑥}_{𝑖-1}𝑑{𝑥}_{𝑖+1}\dots 𝑑{𝑥}_{𝑛}$

Uniformly Distributed Vector

The joint density function is given by

$𝑓(𝑥,𝑦)=\frac{1}{(𝑏-𝑎)(𝑑-𝑐)}{𝕀}_{(𝑎,𝑏]}(𝑥){𝕀}_{(𝑐,𝑑]}(𝑦)$

Two Dimensional Normal

The mapping which is given by

$𝑓(𝑥,𝑦)=\frac{1}{2𝜋}{𝑒}^{-\frac{{𝑥}^2+{𝑦}^2}{2}}$

This is for both variables being (N(0, 1)).

If we want to prove that a function is a density, we need to prove only positivity and equivalence to one on the integral over the interval.

Change of Variable

Let’s take a random vector X with density ${𝑓}_{𝑋}$, and $𝑌=𝜑(𝑋)$ where this function is a multivariate function that transforms X. The density of Y is, in the most general case,

${𝑓}_{𝑌}(𝑦)={𝑓}_{𝑋}({𝜑}^{-1}(𝑦))\left|det(𝐽({𝜑}^{-1}(𝑦)))\right|$

J the Jacobian is a matrix of partials, from

$\frac{𝜕{𝜑}_1}{𝜕{𝑥}_1}\dots \frac{𝜕{𝜑}_1}{𝜕{𝑥}_{𝑛}}$

going left to right

You’re kind of stacking these multivariate functions on top of each other:

$𝜑=({𝜑}_1({𝑥}_1,\dots ,{𝑥}_{𝑛}),\dots ,{𝜑}_{𝑛}({𝑥}_1,\dots ,{𝑥}_{𝑛}))$

To get the inverse of $𝜑$, you treat it as a system

$\{𝑢=𝑥𝑦𝑣=\frac{𝑥}{𝑦}\to 𝑢𝑣={𝑥}^2\to 𝑥=\sqrt{𝑢𝑣},𝑦=\sqrt{\frac{𝑢}{𝑣}}$

${𝜑}^{-1}(𝑢,𝑣)=(\sqrt{𝑢𝑣},\sqrt{\frac{𝑢}{𝑣}})$

Then you can get the Jacobian from the vector

$\frac{𝜕}{𝜕𝑢}\sqrt{𝑢𝑣}\frac{𝜕}{𝜕𝑣}\sqrt{𝑢𝑣}\frac{𝜕}{𝜕𝑥}\sqrt{\frac{𝑢}{𝑣}}\frac{𝜕}{𝜕𝑣}\sqrt{\frac{𝑢}{𝑣}}$

Then calculate each partial, then take the determinant with a calculator.

Convolution

The convolution will allow you to find the density of the sum of random variables.

$(𝑓\ast 𝑔)(𝑥)={\int }_{ℝ}𝑓(𝑦)𝑔(𝑥-𝑦)𝑑𝑦={\int }_{ℝ}𝑔(𝑧)𝑓(𝑥-𝑧)𝑑𝑧$

Take the convolution of the two density functions to get the result.

Examples

If you’re determining pairwise independence, you can use the procedure described earlier.

But for mutual independence, you will need to add all them together