Joint and Conditional Distributions

Lecture Notes

(came late…)

Example 7.8

(a)

Keep in mind that you can’t separate $𝜖$ out from $𝑌$ because they are not necessarily independent.

We can use the fact that $𝜖$ is complementary to divide up what $𝑋$ means.

$\mathbb{P}(𝑋\le 𝑥)\mathbb{P}(𝜖=1)+\mathbb{P}(-𝑋\le 𝑥)\mathbb{P}(𝜖=-1)$

But notice that those two X densities are symmetric! Geometrically, they mirror each other the normal density. And so we can say that these two are equal.

$=\mathbb{P}(𝑋\le 𝑥)$

And there we go! That’s the same as X, so we can say that

$𝑌=𝑁(0,1)$

What about the joint distribution?

We can create a similar negative contrast.

Open question: work through the joint distribution exercise.

(c)

Since both X and Y are normal, the expectation is 0.

$\text{Cov}(𝑋,𝑌)=𝔼[𝑋𝑌]-𝔼[𝑋]𝔼[𝑌]=𝔼[𝑋,𝑌]$

We can make this easier by splitting this into two situations.

$=𝔼[𝑋\cdot 𝜖𝑋]=𝔼[𝜖]𝔼[{𝑋}^2]=0$

The conclusion is that “X and Y are uncorrelated” because their covariance is equal to 0.

(d)

But they are not independent! This is a great counterexample to the false intuition that variables that are uncorrelated are necessarily independent. That’s why it’s not enough to calculate the covariance to determine independence.

$𝔼[{𝑋}^2{𝑌}^2]=𝔼[{𝑋}^2{𝜖}^2{𝑋}^2]=𝔼[{𝜖}^2{𝑋}^4]=𝔼[{𝜖}^2]𝔼[{𝑋}^4]=1\times 3=3$

$𝔼[{𝑋}^2]=1$

$𝔼[{𝑌}^2]=𝔼[{𝜖}^2{𝑋}^2]=𝔼[{𝜖}^2]𝔼[{𝑋}^2]=1$

And that indicates

$𝔼[{𝑋}^2{𝑌}^2]\ne 𝔼[{𝑋}^2]𝔼[{𝑌}^2]$

And that is a proof by contradiction!

Example 7.12

(a)

In order to prove that this is a density, we need to prove that that the area under its function (integral) is equal to 1 over the entire domain.

We need to do a double integral here because the domain is across two variables.

The result becomes the integral over the exponential distribution which is known to be 1.

(b)

We are going to take some basic properties. What is the marginal distribution for each variable?

${𝑓}_{𝑌}(𝑦)={𝑎}^2𝑦{𝑒}^{-𝑎𝑦}$

And we can recognize they take the form

${𝑓}_{𝑌}(𝑦)={𝜆}^{𝑘}{𝑒}^{-𝜆𝑦}/𝑘!$

Example 7.13

(a)

This type of problem is a way to flip around already understood concepts. We still need the same formula for density, but now we’re actually solving for something instead it proving something we already know.

We are taking a sum here because $\mathbb{P}(𝑋=𝑥)$ is the notation used for discrete random variables.

Conditional Distribution: The Discrete Case

We have the marginal probability of which we are already familiar. ${𝑝}_{𝑋}(𝑥)=\mathbb{P}(𝑋=𝑥)={\sum }_{𝑦}\mathbb{P}(𝑋=𝑥,𝑌=𝑦)={\sum }_{𝑦}𝑝(𝑥,𝑦)$

From now on we will use the following notation: ${𝑝}_{𝑋\mid 𝑌}(𝑥\mid 𝑦)$

which is the conditioning random variable.

${𝑝}_{𝑋\mid 𝑌}(𝑥\mid 𝑦)=\mathbb{P}(𝑋=𝑥\mid 𝑌=𝑦)=\frac{\mathbb{P}(𝑋=𝑥,𝑌=𝑦)}{\mathbb{P}(𝑌=𝑦)}=\frac{𝑝(𝑥,𝑦)}{{𝑝}_{𝑌}(𝑦)}$

Conditional CDF

${𝐹}_{𝑋\mid 𝑌}(𝑥\mid 𝑦)=\mathbb{P}(𝑋\le 𝑥\mid 𝑌=𝑦)={\sum }_{𝑡\le 𝑥}\mathbb{P}(𝑋=𝑡\mid 𝑌=𝑦)$

$={\sum }_{𝑡\le 𝑥}{𝑝}_{𝑋\mid 𝑌}(𝑡\mid 𝑦)$

We can use all of these expressions, which are the same. They might be useful in different contexts, though.

8.16

Conditional Expectation

$𝔼[𝑋\mid 𝑌=𝑦]={\sum }_{𝑥}𝑥\mathbb{P}(𝑋=𝑥\mid 𝑌=𝑦)={\sum }_{𝑥}𝑥{𝑝}_{𝑋\mid 𝑌}(𝑥\mid 𝑦)$

And this expectation generalizes across any function of X.

This is the same result of linear expectation.

8.19

The joint density is not actually easy to determine here.

What is the probability that (Y = 3)? It’s a geometric random variable, based on our knowledge of the “trials until you have success” property of geometric random variables.

$\frac{(𝑥-1{)}^2}{{𝑥}^2}$

Let’s see the probability for when (Y = 3). We will take the probability of every value of (x).

We actually have a restriction on our domain here.

There’s a common identity we can use for the geometric distribution:

$𝑔(𝑥)={\sum }_{𝑛=0}^{\infty }{𝑥}^{𝑛}=\frac{1}{1-𝑥}$

${𝑔}^{\prime }(𝑥)={\sum }_{𝑛=1}^{\infty }𝑛{𝑥}^{𝑛-1}=\frac{1}{(1-𝑥{)}^2}$

$𝑥{𝑔}^{\prime }(𝑥)={\sum }_{𝑛=0}^{\infty }𝑛{𝑥}^{𝑛}=\frac{𝑥}{(1-𝑥{)}^2}$