Gamma, Beta, and Chi-Squared Distributions

Week 8

Lecture Notes

Chi-Squared

Definition 5.36 says that X is a Chi squared distribution whose pdf is described by

$𝑓(𝑥)=\frac{1}{2^{𝑛/2}\mathrm{\Gamma }(\frac{𝑛}{2})}{𝑥}^{\frac{𝑛}{2}-1}{𝑒}^{-\frac{𝑥}{2}}$

where $𝑋\sim {𝑋}_{𝑛}^2$

This is actually linked to the gamma distribution

$𝑋\sim {𝑋}_{𝑛}^2$, then $𝑋\sim \mathrm{\Gamma }(\frac{𝑛}{2},\frac{1}{2})$

There is also a remark which says that the gamma distribution has an additive property. And if you inverse a gamma distribution, you get the inverse gamma distribution.

Beta Distribution

Described by 5.4.5

Gamma distribution is defined under the gamma function. We will need a two variable beta function. The intermediate function is defined as

$\mathrm{B}(𝑎,𝑏)≔{\int }_0^1{𝑥}^{𝑎-1}(1-𝑥{)}^{𝑏-1}𝑑𝑥$

The integral is univariate. There’s an apparent symmetry. If we do a change of variable of $𝑦=1-𝑥$ then

$\mathrm{B}(𝑏,𝑎)={\int }_0^1{𝑥}^{𝑏-1}{𝑦}^{𝑎-1}𝑑𝑥$

and

$={\int }_1^0(1-𝑦{)}^{𝑏-1}{𝑦}^{𝑎-1}(-𝑑𝑦)$

Which when you cancel out the negative, they end up being equal. And in fact it can be shown that

$\mathrm{B}(𝑎,𝑏)=\mathrm{B}(𝑏,𝑎)$

and a more useful result proposition 5.39, which is that

$\mathrm{B}(𝑎,𝑏)=\frac{\mathrm{\Gamma }(𝑎)\mathrm{\Gamma }(𝑏)}{\mathrm{\Gamma }(𝑎+𝑏)}$

this will be very useful later. Proving this is very tedious calculus.

We can now introduce the beta distribution through definition 5.40. If $𝑥$ has the following pdf

${𝑓}_{𝑎,𝑏}(𝑥)=\frac{{𝑥}^{𝑎-1}(1-𝑥{)}^{𝑏-𝑎}}{\mathrm{B}(𝑎,𝑏)}{𝐼}_{[0,1]}(𝑥)$

then it’s in the beta distribution.

Proposition 5.41 says that the integral is 1.

Proposition 5.42 tells us the expected value and variance.

$𝔼[𝑋]=\frac{𝑎}{𝑎+𝑏}$

$𝕍[𝑋]=\frac{𝑎𝑏}{(𝑎+𝑏{)}^2(𝑎+𝑏+1)}$ To prove this, we take the definition of the expectation

$𝔼[𝑋]={\int }_0^1𝑥{𝑓}_{𝑎,𝑏}(𝑥)𝑑𝑥$

$=\frac{1}{\mathrm{B}(𝑎,𝑏)}{\int }_0^1{𝑥}^{𝑎}(1-𝑥{)}^{𝑏-1}𝑑𝑥$

We can then adjust the $𝑎$ to make it agree with the beta function

$=\frac{\mathrm{B}(𝑎+1,𝑏)}{\mathrm{B}(𝑎,𝑏)}$

Now I can adjust it to be the gamma function instead.

$=\frac{\mathrm{\Gamma }(𝑎+𝑏)}{\mathrm{\Gamma }(𝑎)\mathrm{\Gamma }(𝑏)}\cdot \frac{\mathrm{\Gamma }(𝑎+1)\mathrm{\Gamma }(𝑏)}{\mathrm{\Gamma }(𝑎+𝑏+1)}$

We can cancel out some things, and then utilize the property of the gamma function that $\mathrm{\Gamma }(𝑛+1)=𝑛\mathrm{\Gamma }(𝑛)$

$\frac{𝑎}{𝑎+𝑏}$

Then for variance, skipping some steps, we can show that

$𝔼[{𝑋}^2]=\frac{1}{\mathrm{B}(𝑎,𝑏)}{\int }_0^1{𝑥}^{𝑎+1}(1-𝑥{)}^{𝑏-1}𝑑𝑥$

Using a similar method, we can rewrite this as

$=\frac{\mathrm{B}(𝑎+2,𝑏)}{\mathrm{B}(𝑎,𝑏)}$

Now we have from gamma

$=\frac{\mathrm{\Gamma }(𝑎+2)\mathrm{\Gamma }(𝑏)}{\mathrm{\Gamma }(𝑎+𝑏+2)}\cdot \frac{\mathrm{\Gamma }(𝑎+𝑏)}{\mathrm{\Gamma }(𝑎)\mathrm{\Gamma }(𝑏)}$

canceling and substituting with our values, it’s

$=\frac{(𝑎+1)𝑎}{(𝑎+𝑏+1)(𝑎+𝑏)}$

Then to calculate variance, we get

$=\frac{(𝑎+1)𝑎}{(𝑎+𝑏+1)(𝑎+𝑏)}-\frac{{𝑎}^2}{(𝑎+𝑏{)}^2}$

which will simplify into our answer