Continuous Random Variables and Transformations
Lecture Notes
Recall for a discrete random variable. The moment formula is
The moment for a continuous rv is the integral
All the possibilities are the real line here.
Most of today’s class will deal with integrals. We are going to calculate a lot of these, just like a single variable calculus class. We will also learn multivariable calculus for multiple variables.
Generalization of expectation for any function is definition 5.9 (skip technical assumptions)
You would just compute the antiderivative of and then evaluate the integral by subtracting the top from bottom. For an indicator variable, it is 0 to 1 not -infinity to infinity.
Now I want to go further, I want to know the full distribution, not just the expectation, of
This is given by theorem 5.10 which tells us how to determine the probability density function. The PDF of is given by
Here, denotes the composition of f on inverse h. First I apply the inverse h, then I apply the f. Sometimes you use the notation
How do you prove this? We will sketch. We will make the assumption that is non-decreasing.
We can do the inverse because it’s non-decreasing.
This is true, so we can write this probability as the integral of the PDF of X
If I want to say the PDF of the random variable , then it gets the subscript; without the subscript, it’s for .
Now we will do a change of variables
We know that is inside the integral, so we will write the equivalence that when it is in between those two values, it is equivalent to the event that is in between and .
This is important because we will change the integration range:
You can move this part to the front
We will see that
Now if we take the derivative of this integral, then we get
We get something like the fundamental theorem of calculus here, or part of it, or Leibniz rule. you have the following result:
This identity is learned in calculus.
So, we can show that everything inside doesn’t depend on , so we can just evaluate the integral with it.
Then we will use this result, from a second theorem of calculus called the implicit function theorem. This is more real analysis, here’s a simple example.
Now the idea is we want to do differentiation on both sides.
Because we have this equality, the derivative preserves the equality (maybe not integration because of range). Applying the chain rule, we get
And then we do a change of variable.
y is just a dummy variable, so we can apply this here, to finally show that
Interesting corollary 5.11 that if X has pdf f and we have a Y which is aX + b, then pdf of Y is
This is a special case when the is , can be shown by differentiation. All you need to find is and
We can essentially think of as a linear combination of a different random variable whose pdf is known, and then we can derive the pdf from the existing pdf f.
5.4.1
We are concerned with a uniform random variable defined continuously on on
Definition 5.13 says that the pdf of this is
We will use the notation to indicate uniform distribution
Let’s do something non-trivial. Proposition 5.15 says that if we have , then
How do we prove? We can create the integral based on the definition
We have from algebra that
which we can use to prove the variance.
We are generally solving for
and that’s how we solve for expectation, for being or
We have to turn these types of problems into a pdf, that will be done on the midterm.
What is the cdf for a uniform variable?
By proposition 5.16, it’s
This is the same as
We can accomplish this case conclusion by the integral
Very simple proof for this.
Now let’s consider a simple example 5.8 where , we will compute
To solve, this we will solve as an integral
This is one way to solve this, or you could separate it into cdf up to ¾ and ¼.
5.4.2
The next thing is the exponential distribution, definition 5.17. We introduce the auxiliary parameter , then
That is what is meant when we are given a distribution. How do we verify that this result is a pdf? Proposition 5.18 says
The range of integration starts from 0
Now we can do some calculus
Now we can do integration by parts, which is a very important technique for this chapter that we will do over and over again.
If we are interested in integrating
And this is the process of integration by parts.
(We don’t actually need integration by parts here, but we will need to do it for the moments)
Now we will show the moments by proposition 5.19
We will now try to evaluate this integral
And this is where we need integration by parts
First we take the product
Variance is not easy because you need to integrate twice by parts.
We will try now to get rid of that denominator to get
Integration by parts:
And we already have shown this result so it is
And then after subtracting by the expectation squared you get
What is the cdf of the exponential random variable? Proposition 5.20 will tell us that
Here is the proof, which is just calculation
The difference between a CDF and a statistical measure is that we’re measuring up to (x), not infinity, but the technique is the same.
There is a memoryless property of the exponential distribution.
I know that my exponential variable is bigger than t. There is some interval between t and t + s which is length s. I know my random variable is bigger than t + s, given that it is bigger than t. But actually it doesn’t really matter what the probability is between t and t + s, there is no memory of the starting point. You only care about the probability after s. (We will not prove this)
Let’s take example 5.9, it will be easy to prove the cdf
Then we get expectation by the formula How do we solve if we are given probability between an interval like
We take the integral between these two points.
And we can actually do a change of variable here from
Which will give us
5.4.3
The normal distribution by definition 5.22 has the distribution function
denoted by
We have to show that the normal distribution is a PDF, which is not very straightforward.
Proposition 5.24
Proposition 5.25 tells us that the expected value and variance are
which is very convenient for us. But we will need to formally verify this
We will copy this pdf into the integral
And we are going to do a change of variable
We have the same integration region because they are infinite. We can replace x in this whole function.
We will separate the integral into two parts
The first integrand is an odd function. This means that . Even functions are . For any integral from -infinity to infinity, we can combine all the negatives and all the positives. And if it is symmetric around 0 with the opposite, which is the meaning of odd, then when they’re added it should give you 0.
Then we can recognize that for the second integrand that it must be equal to by the fact that the normal distribution area is equal to 1 (assumed).
Now for the variance. By definition, because we know the expectation already, we get
which by the pdf is
Then we can do a change of variable, and then solve via polar coordinates on page 138
Remark 5.2.7 says that if , then
The inverse operation
is known as standardization.
Proof: if you let
Using the corollary 5.11 for linear combination we can prove this.
Useful property of normal distribution. If we have two normal distributions and that are independent, then
We have to do a convolution to solve this, involves integration between the two things.
5.4.14
Last one for today is the gamma distribution.
We need some preparation, in the gamma function.
Proposition 5.30 says the properties of the gamma function. We have the property (a) that
and (b) that
and (c), which the source note does not prove,
Proof of (a) is that
We can separately calculate by the power rules.
The technique is to do the integration by parts and factor out certain quantities.
If we want to find out (b), then we can do induction once we find
What about the pdf? Given by definition 5.31
It has a lot of weird special cases.
Proposition 5.32 shows that this is a true density function. We will do integration from 0 to the infinity of this density
After we cancel out the
which is just the definition of gamma
Proposition 5.33 says for E and V that
We know from the definition of expectation and combining similar terms that
Let’s do a change of variable to
Taking this out, I get
Here we can use the result from before of the definition of gamma to substitute the integral.
To get variance, we have
Change of variable…
I can sequentially use the value of a to get this result.
Finally, proposition 5.35 shows us that for , then
Our proof is that
x is a normal density, so we can write this as
now we’re going to differentiate on both sides
. The source note starts the derivative calculation here; the remaining algebra is not recorded.
These two minus signs cancel out and you get both
And the two will cancel out and you can put this downstairs
Then you can rewrite this exactly as
And then we can know from our definition that
Using the progressive values of gamma makes it easier to do calculations with them.