Student's t and Heavy-Tailed Distributions

Student T-distribution

Definition 5.4.6 says that the pdf is

$𝑓(𝑥)=\frac{\mathrm{\Gamma }(\frac{(𝑛+1)}{2})}{\sqrt{𝜋𝑛}\mathrm{\Gamma }(\frac{𝑛}{2})}{\left(1+\frac{{𝑥}^2}{𝑛}\right)}^{-\frac{𝑛+1}{2}}$

then X is a Student t-distribution with degrees of freedom ${𝑡}_{𝑛}$.

Proposition 5.48 will tell us all the moments, which are not easy to prove.

$𝔼[𝑋]=0$ if $𝑛>1$, and

$𝕍[𝑋]=\frac{𝑛}{𝑛-2}$ if $𝑛>2$

This distribution has a heavier tail than normal, but not as heavy as the next

Heavy Tailed Distribution

Definition 5.4.9 tells us the Pareto distribution which describes the power law in natural sciences. Income of people follows a heavy tail, there are always people earning a lot of money. Benford law erlates to cryptography, Zipf law relates to economics.

$𝑓(𝑥)=\frac{𝑎{𝑏}^{𝑎}}{{𝑥}^{𝑎+1}}{𝐼}_{[𝑏,\infty ]}(𝑥)$

I’m only looking at the value above b which is a positive quantity, it’s decaying at a power of a + 1. A larger a means lighter tails.

Proposition 5.50 says that if we integrate this, you get 1, so this is a density function.

Proposition 5.51 tells us the moments:

$𝔼[𝑋]=\frac{𝑎𝑏}{𝑎-1}$

$𝕍[𝑋]=\frac{𝑎{𝑏}^2}{(𝑎-1{)}^2(𝑎-2)}$

There is a link between the Pareto and exponential distribution. This is proposition 5.53.

If you have a exponential distribution

$𝑍\sim \text{Exp}(𝑎)$, then for a $𝑏>0$, then

$𝑋=𝑏\cdot {𝑒}^{𝑍}$ is a Pareto distribution $𝑃(𝑎,𝑏)$.

Theorem 5.10

Log-Normal Distribution

Definition 5.54. The Black-Scholes model follows this distribution.

It’s called log normal because the log of it is normal

$𝑓(𝑥)=\frac{1}{𝑥𝜎\sqrt{2𝜋}}{𝑒}^{-\frac{(ln𝑥-𝜇{)}^2}{2{𝜎}^2}}{𝐼}_{(0,\infty )}(𝑥)$

then $𝑋\sim \text{LogNormal}(𝜇,{𝜎}^2)$

We can also describe the cdf of the distribution:

${𝐹}_{𝑋}(𝑡)=\mathbb{P}(𝑋\le 𝑡)=\mathrm{\Phi }\left(\frac{ln𝑡-𝜇}{𝜎}\right)$

where $\mathrm{\Phi }$ is the cdf of the normal distribution.

Proposition 5.55 will tell us the density proof, will be on final

Proposition 5.56 tells us the moments:

$𝔼[𝑋]={𝑒}^{𝜇+\frac{1}{2}{𝜎}^2}$

$𝕍[𝑋]=({𝑒}^{{𝜎}^2}-1){𝑒}^{𝑡𝜇+{𝜎}^2}$

Laplace Distribution

Definition 5.59

$𝑓(𝑥)=\frac{𝜃}{2}{𝑒}^{-𝜃|𝑥|}$

Something is different is that it’s the absolute value of $𝑥$, and this applies to the whole real line.

Proposition 5.60 proves the density is 1

Proposition 5.61 tells us the moments

$𝔼[𝑋]=0$

the proof:

$𝔼[𝑋]=\frac{𝜃}{2}{\int }_{-\infty }^{\infty }𝑥{𝑒}^{-𝜃|𝑥|}𝑑𝑥$

Because this is an odd function, we can say that the expectation must be 0.

$𝕍[𝑋]=\frac{1}{𝜃}$

Double Exponential Distribution

5.4.10

$𝑓(𝑥)=\{𝑝{𝛼}_1{𝑒}^{-{𝛼}_1𝑥},𝑥>0(1-𝑝){𝛼}_2{𝑒}^{{𝛼}_2𝑥},𝑥\le 0$

If x is positive, it’s exponential, if it’s negative, it follows a different exponential distribution

Examples

These come from the second textbook G, where homework is.

5.1

the pdf

$𝑓(𝑥)=\frac{2}{𝜋(1+{𝑥}^2)}{𝕀}_{[0,\infty )}(𝑥)$

show that it’s a density function and that the expected value does not exist

We need to solve for the integral over the range to be 1.

${\int }_0^{\infty }\frac{2}{𝜋(1+{𝑥}^2)}𝑑𝑥=\frac{2}{𝜋}{\int }_0^{\infty }\frac{1}{1+{𝑥}^2}𝑑𝑥$

And that’s just arctan

Then you get one by solving for arctan.

For expectation, the proof is done by u-substitution for x^2 + 1, and then you can show that the expectation is infinite, which means it doesn’t exist.

5.2

If $𝑋\sim \text{Unif}[0,1]$, show that ${𝑋}^2\sim \text{Beta}(\frac{1}{2},1)$

We will use theorem 5.10 for solving for pdf in order to get ${𝑋}^2$ from the uniform. We can use ${𝑥}^2$ as our function because under the interval $[0,1]$ it’s monotonic increasing.

We need to find the inverse of our function, and the derivative of our function. That’s

${ℎ}^{-1}(𝑦)=\sqrt{𝑦}$

${ℎ}^{\prime }(𝑥)=2𝑥$

$\frac{1}{2\sqrt{𝑦}}$

Now we have to prove that this is the density function for beta.

We can substitute the numbers given to us in the beta formula, and then use the gamma decomposition to give us actual numbers.

5.3

We want to show that $𝑌={𝑋}^2$ has the density function The density function is the derivative of the cdf, so if you want to show the pdf

4.7

cdf of a discrete random variable

$𝐹(𝑥)=\{0𝑥<0𝑥/40\le 𝑥<11/21\le 𝑥<21/12𝑥+1/22\le 𝑥<31𝑥\ge 3$

How do you compute the probabilities?

$\mathbb{P}(𝑋<2)=\frac{1}{2}$ We can directly compute this easily. What about

$\mathbb{P}(𝑋=2)=\mathbb{P}(𝑋\le 2)-\mathbb{P}(𝑋<2)$

This is because it’s a cdf and you’re getting rid of all that came before which is actually under a different function. Same for

$\mathbb{P}(1\le 𝑋<3)=\mathbb{P}(𝑋<3)-\mathbb{P}(𝑋<1)$

What about the greater than? We can use the complement.

$\mathbb{P}(𝑥>\frac{3}{2})=1-\mathbb{P}(𝑥\le \frac{3}{2})$

So if you’re doing equal, you get rid of everything up to that point. For a point in an interval this is actually 0 which makes sense because the probability of any individual point is usually 0.

$\mathbb{P}(𝑥=\frac{5}{2})=\mathbb{P}(𝑥\le \frac{𝑡}{2})-\mathbb{P}(𝑋<\frac{5}{2})=0$

4.8

If we flip a coin twice, and X is the number of tails, what is the cdf of X?

Let’s calculate

$𝐹(𝑡)=\mathbb{P}(𝑋\le 𝑡)=\{0𝑡<0\mathbb{P}(𝑥=0)0\le 𝑡<1\mathbb{P}(𝑥=0\cup 𝑥=1)1\le 𝑡<21𝑡\ge 2$

4.10

if we consider cdf of X

$𝐹(𝑡)=\{0𝑡<0\frac{1}{2}{𝑡}^{2}0\le 𝑡<1𝑘(4𝑡-{𝑡}^2)1\le 𝑡<21𝑡\ge 2$

In order to consider the independence of two sets

$𝐴=\left\{\frac{1}{2}\le 𝑥\le \frac{3}{2}\right\}$

$𝐵=\left\{1\le 𝑥\right\}$

We want the probability of the intersection of those sets to be the same as the product of their probabilities. In this case, they are not, so they are not independent.

5.18

Deck of 52 cards. Draw the cards with replacement until an ace is drawn. Calculate the probability that at least 10 draws are needed to get the first ace.

If you see “until the first” it means, “until the first success event”, which means the geometric distribution.

This is the $𝑋\sim 𝐺𝑒𝑜(\frac{1}{13})$

Then we can get the pmf as

$\mathbb{P}(𝑋=𝑛)={\left(\frac{12}{13}\right)}^{𝑛-1}\frac{1}{13}$ How do you do at least ten draws?

$\mathbb{P}(𝑋\ge 10)={\sum }_{𝑛=10}^{\infty }{\left(\frac{12}{13}\right)}^{𝑛-1}\frac{1}{13}$

$=\frac{1}{13}{\sum }_{𝑛=10}^{\infty }$

We will do a change of variable of $𝑘=𝑛-10$

$=\frac{1}{13}{\sum }_{𝑘=0}^{\infty }{\left(\frac{12}{13}\right)}^{𝑘+9}$

$=\frac{1}{13}{\left(\frac{12}{13}\right)}^9{\sum }_{𝑘=0}^{\infty }{\left(\frac{12}{13}\right)}^{𝑘}$

VERY IMPORTANT FORMULA FOUND HERE:

${\sum }_{𝑛=0}^{\infty }{𝑝}^{𝑛}=\frac{1}{1-𝑝}$

We will use this to solve geometric distributions.

Now let’s look at continuous time

6.8

Typical example: we are given the density and are asked to solve for the moments.

$𝑓(𝑥)=\frac{27}{490}(𝑒{𝑥}^2-2𝑥),\frac{2}{3}<𝑥<3$

$𝔼[𝑋]={\int }_{-\frac{2}{3}}^3𝑥\cdot \frac{27}{490}(3{𝑥}^2-2𝑥)𝑑𝑥$

We can solve this integral which should give us the answer 283/120. Left as exercise to reader.

7.7

Check notes…

6.3

$𝑓(𝑥)=\frac{2}{{𝑥}^2},1<𝑥<2$

Find the distribution and pdf of $𝑌={𝑋}^2$

same as the uniform question.

There are many ways to solve this. Theorem 5.10 is one way to solve it. You can also start from the basic principle of how cumulative distribution works

$𝐺(𝑡)=\mathbb{P}(𝑌\le 𝑡)=\mathbb{P}({𝑋}^2\le 𝑡)=\mathbb{P}(-\sqrt{𝑡}\le 𝑥\le \sqrt{𝑡})=\mathbb{P}(𝑥\le \sqrt{𝑡})=\mathbb{P}(𝑥\le \sqrt{𝑡})-\mathbb{P}(𝑋\le -\sqrt{𝑡}$

The last probability we canceled so we are only concerned with the middle probability,

$\mathbb{P}(1\le 𝑋\le \sqrt{𝑡})$

We have to integrate to get this

$={\int }_1^{\sqrt{𝑡}}\frac{2}{{𝑥}^2}𝑑𝑥=-\frac{2}{𝑥}{|}_{𝑥=1}^{𝑥=\sqrt{𝑡}}=-\frac{2}{\sqrt{𝑡}}+2$

Then we can find that $𝐺(𝑡)$ is substituted this.

Then we can find the density function, which is known to be the derivative of the cdf from earlier:

$𝑔(𝑡)={𝐺}^{\prime }(𝑡)$

That’s it from the examples.