π Lecture 12 β A Linear Combination of PDFs is a PDF
π Overview: This lecture establishes that a linear combination of probability density functions (PDFs), where the coefficients are nonnegative and sum to 1, is itself a valid probability density function. It also proves that the mean of such a linear combination equals the same linear combination of the individual means. This concept is fundamental for constructing mixture distributions.
ποΈ Topics Covered
The lecture introduces the concept that a linear combination of continuous-type PDFs (with nonnegative coefficients summing to 1) produces a new PDF. It then proves this by showing the integral of the weighted sum equals 1. Next, the lecture proves that the expected value (mean) of this new distribution is a linear combination of the individual means. The key result is that both the PDF and its mean preserve the linear combination property.
π Lecture Summary
Consider k continuous-type distributions
Let there be k continuous-type distributions, each with its own probability density function ( f_i(x) ) and mean ( \mu_i ), for ( i = 1, 2, \ldots, k ). Let ( c_i \ge 0 ) be constants for ( i = 1, 2, \ldots, k ), such that ( c_1 + c_2 + \ldots + c_k = 0 + \ldots + 0 = 1 ). Define a new function ( w(x) = c_1 f_1(x) + c_2 f_2(x) + \ldots + c_k f_k(x) ). The lecture aims to show that ( w(x) ) is a valid PDF.
π Definition β Linear combination: An expression constructed from a set of terms by multiplying each term by a constant and adding those results. For example, ( ax + by ) is a linear combination of ( x ) and ( y ).
Proof: A Linear Combination of PDFs is a PDF
To prove that ( w(x) = c_1 f_1(x) + c_2 f_2(x) + \ldots + c_k f_k(x) ) is a valid PDF, we must show its integral over all real numbers equals 1. The left-hand side of the integral is: [ \int_{-\infty}^{\infty} w(x) dx = \int_{-\infty}^{\infty} [c_1 f_1(x) + c_2 f_2(x) + \ldots + c_k f_k(x)] dx ] Since integration is a linear operator, this becomes: [ = c_1 \int_{-\infty}^{\infty} f_1(x) dx + c_2 \int_{-\infty}^{\infty} f_2(x) dx + \ldots + c_k \int_{-\infty}^{\infty} f_k(x) dx ] Since each ( f_i(x) ) is a PDF, the integral of each is 1: ( \int_{-\infty}^{\infty} f_i(x) dx = 1 ). Therefore: [ = c_1 (1) + c_2 (1) + \ldots + c_k (1) ] [ = c_1 + c_2 + \ldots + c_k ] By our choice of constants, ( c_1 + c_2 + \ldots + c_k = 1 ). Hence, the left-hand side equals the right-hand side: [ \int_{-\infty}^{\infty} w(x) dx = 1 ] This proves ( w(x) ) is a valid PDF. In other words, the mean of a linear combination of PDFs is that same linear combination of the means of those PDFs.
π Formula: ( w(x) = c_1 f_1(x) + c_2 f_2(x) + \ldots + c_k f_k(x) ) β A weighted sum of PDFs, where weights are nonnegative and sum to 1, creating a new PDF.
Show that the mean of the distribution having pdf ( w(x) ) is ( \sum_{i=1}^{k} c_i \mu_i )
The mean (expected value) of the new distribution with PDF ( w(x) ) is: [ E[X] = \int_{-\infty}^{\infty} x w(x) dx = \int_{-\infty}^{\infty} x [c_1 f_1(x) + c_2 f_2(x) + \ldots + c_k f_k(x)] dx ] Again, using linearity of integration: [ = c_1 \int_{-\infty}^{\infty} x f_1(x) dx + c_2 \int_{-\infty}^{\infty} x f_2(x) dx + \ldots + c_k \int_{-\infty}^{\infty} x f_k(x) dx ] The expected value of ( X ) for each individual distribution is ( \int_{-\infty}^{\infty} x f_i(x) dx = \mu_i ). Therefore: [ = c_1 \mu_1 + c_2 \mu_2 + \ldots + c_k \mu_k ] Hence, we get: [ E[X] = \sum_{i=1}^{k} c_i \mu_i ]
π‘ Why this matters: This property allows us to model complex distributions by mixing simpler ones. For example, if we have data that appears to come from two different populations (like heights of men and women), we can represent the overall distribution as a weighted average of the two individual distributions.
π Formula: ( E[X] = \sum_{i=1}^{k} c_i \mu_i ) β The mean of a linear combination of PDFs is the same linear combination of their individual means.
π Example: Given three PDFs with means ( \mu_1 = 5 ), ( \mu_2 = 10 ), ( \mu_3 = 15 ) and coefficients ( c_1 = 0.5 ), ( c_2 = 0.3 ), ( c_3 = 0.2 ) (note: ( 0.5 + 0.3 + 0.2 = 1 )), the mean of the mixture distribution ( w(x) ) is: [ E[X] = 0.5(5) + 0.3(10) + 0.2(15) = 2.5 + 3.0 + 3.0 = 8.5 ]
β Key Takeaways
The most critical concept is that a linear combination of PDFs, with nonnegative weights summing to 1, is itself a valid PDF. This is proven by showing its total integral equals 1. Additionally, the mean (expected value) of this weighted PDF is the same weighted sum of the individual means. The conditions for the constants ( c_i ) being nonnegative and summing to 1 are essential for the resulting function to be a legitimate probability distribution. This principle allows us to construct and analyze mixture distributions by combining simpler component distributions.
π§ Quick Revision Questions
- What are the two conditions the constants ( c_i ) must satisfy for ( w(x) = \sum c_i f_i(x) ) to be a PDF?
- Explain why the integral of ( w(x) ) equals 1.
- How is the mean of a mixture distribution related to the means of its component distributions?
- If ( w(x) = 0.7 f_1(x) + 0.3 f_2(x) ) and ( \mu_1 = 20, \mu_2 = 50 ), what is the mean of ( w(x) )?
- Why is the property of linearity of integrals crucial in proving both results in this lecture?
π Lecture 13 β Concept of Cumulative Distribution Function (discrete and continuous)
π Overview: This lecture introduces the Cumulative Distribution Function (CDF), a fundamental concept in probability theory that describes the probability that a random variable takes a value less than or equal to a specific point. The lecture covers the definition, properties, and graphical representation of CDFs for both discrete and continuous random variables, with practical examples to illustrate the differences.
ποΈ Topics Covered
The lecture begins with the formal definition of the Cumulative Distribution Function (CDF) for any random variable. It then explains the graphical representation of CDFs, noting that for discrete random variables the CDF forms a step function (staircase), while for continuous random variables it forms a continuous curve or line. Two detailed examples are presented: one for a discrete random variable with a probability mass function (pmf) and another for a continuous random variable uniformly distributed between 0 and 2, showing how to derive and sketch the CDF in each case.
π Lecture Summary
Definition: (Cumulative Distribution Function)
Let X be a random variable. Its cumulative distribution function (cdf) is defined by Fβ(x), given by:
Fβ(x) = P(X β€ x) for -β < x < β
As above, we shorten: Fβ(x) = P(X β€ x)
Also, Fβ(x) is often called simply the distribution function (df).
The graph for the discrete case is like a staircase, which is also called a Step function. On the other hand, in the continuous case it is a continuous curve or a line.
π Definition β Cumulative Distribution Function (CDF): The function Fβ(x) = P(X β€ x) that gives the probability that the random variable X takes a value less than or equal to x.
Example 1: Discrete Random Variable
Let pβ(x) be the pmf of a random variable X. Find the cdf F(x) of X and sketch its graph with that of pβ(x) if: a) pβ(x) = 1/3, x = -1, 0, 1, zero elsewhere.
The probability mass function assigns equal probability of 1/3 to each of the three values: -1, 0, and 1.
π Example: For X with pmf pβ(-1) = 1/3, pβ(0) = 1/3, pβ(1) = 1/3:
- For x < -1: F(x) = 0
- For -1 β€ x < 0: F(x) = P(X β€ -1) = 1/3
- For 0 β€ x < 1: F(x) = P(X β€ 0) = P(X = -1) + P(X = 0) = 1/3 + 1/3 = 2/3
- For x β₯ 1: F(x) = P(X β€ 1) = 1/3 + 1/3 + 1/3 = 1
The graph is a step function with jumps at x = -1, 0, and 1.
Figure 1: Distribution Function - shows the staircase pattern for discrete case.
Example 2: Continuous Random Variable
Let X denote a real number chosen at random between 0 and 2. We now obtain the cdf of X.
First, if x < 0, then P(X β€ x) = 0. Next, if x β₯ 2, then P(X β€ x) = 1. Finally, if 0 < x < 2, it follows from equation (2) that P(X β€ x) = P(0 < X β€ x) = x - 0 = x.
Hence the cdf of X is:
Fβ(x) =
- 0, if x < 0
- x/2, if 0 β€ x < 2
- 1, if x β₯ 2
Note: For a uniform distribution on interval [a, b], P(a < X < b) = (b - a) / (b - a) = 1, and for any subinterval [c, d] within [a, b], P(c < X < d) = (d - c) / (b - a).
π Formula: For a uniform distribution on [a, b]: P(c < X < d) = (d - c) / (b - a) for a < c < d < b.
π Example: For X uniformly distributed on [0, 2]:
- Fβ(0.5) = P(X β€ 0.5) = 0.5/2 = 0.25
- Fβ(1) = 1/2 = 0.5
- Fβ(1.5) = 1.5/2 = 0.75
Figure 2: Distribution Function - shows the continuous linear graph from (0,0) to (2,1).
π‘ Why this matters: The CDF is a complete description of a random variable's probability distribution. Unlike the pmf or pdf, the CDF is defined for all real numbers and always exists for any random variable, making it a universal tool for probability calculations.
β Key Takeaways
The Cumulative Distribution Function Fβ(x) = P(X β€ x) is defined for all real numbers and always ranges from 0 to 1 as x increases from negative infinity to positive infinity. For discrete random variables, the CDF is a step function that jumps at each point where the pmf is nonzero, with the jump height equal to the probability at that point. For continuous random variables, the CDF is a continuous function that can be obtained by integrating the probability density function. The CDF is non-decreasing, meaning if xβ < xβ, then F(xβ) β€ F(xβ). The CDF is fundamental because it provides a unified way to compute probabilities: P(a < X β€ b) = F(b) - F(a) for any a < b.
π§ Quick Revision Questions
- What is the formal definition of the Cumulative Distribution Function (CDF)?
- How does the graph of a CDF differ between a discrete random variable and a continuous random variable?
- For the discrete example with pβ(x) = 1/3 at x = -1, 0, 1, what is F(0.5)?
- For a continuous uniform random variable on [0, 2], what is the CDF value at x = 1.5?
- What property must a CDF always satisfy regarding monotonicity?
π Lecture 14 β Example of the CDF of a Discrete Random Variable
π Overview: This lecture demonstrates how to derive and verify the cumulative distribution function (CDF) for a discrete random variable using a concrete example involving drawing slips of paper numbered 1 to 100 from an urn. It shows the step-by-step process of defining the probability mass function and then deriving the CDF formula, validating its correctness.
ποΈ Topics Covered
The lecture covers a single worked example of finding the CDF for a discrete random variable X representing the number on a slip drawn from an urn containing slips numbered 1 through 100, with i slips bearing the number i. It demonstrates the probability mass function (PMF) derivation and the formula for the cumulative distribution function F(x) = x/10100 for 1 β€ x β€ 100.
π Lecture Summary
Example β Setting Up the Problem
Consider an urn containing slips of paper, each with one of the numbers 1, 2, ..., 100. There are i slips with the number i on them for i = 1, 2, ..., 100. For example, there are 25 slips with the number 25. The slips are identical except for the numbers. One slip is drawn at random. Let X be the number on the slip.
The total number of slips in the urn is the sum of integers from 1 to 100: 1 + 2 + 3 + ... + 100 = 5050 slips.
The probability that the drawn slip shows the number x is the number of slips with that number divided by the total number of slips.
π Definition β Probability Mass Function (PMF): For a discrete random variable X, p(x) = P(X = x) = (number of favorable outcomes) / (total number of outcomes)
π Example for PMF: The PMF for this problem is:
- p(1) = 1/5050, p(2) = 2/5050, p(3) = 3/5050, ..., p(100) = 100/5050
- Sum of all probabilities = 5050/5050 = 1 β (confirms it is a valid PMF)
Deriving the CDF
The cumulative distribution function (CDF) F(x) = P(X β€ x) for a discrete random variable is the sum of probabilities for all values less than or equal to x.
For this problem, F(x) = P(X β€ x) = (1 + 2 + 3 + ... + [x]) / 5050, where [x] is the greatest integer less than or equal to x (the floor function).
π Definition β Greatest integer in x ([x]): The largest integer that is less than or equal to x. For example, [25.7] = 25, [50] = 50, [8.1] = 8.
The sum of the first n positive integers is given by the formula: 1 + 2 + 3 + ... + n = n(n+1)/2.
Substituting n = [x]: 1 + 2 + 3 + ... + [x] = x/2
Therefore: F(x) = x / (2 Γ 5050) = x / 10100
π Formula: F(x) = x/10100, for 1 β€ x β€ 100, where [x] is the greatest integer in x.
π‘ Why this matters: This formula allows us to calculate the probability that X takes a value less than or equal to any given x without summing individual probabilities each time.
π Example: To find P(X β€ 25): F(25) = 25/10100 = 25 Γ 26 / 10100 = 650/10100 = 65/1010 = 13/202 β 0.06436
π Example: To find P(X β€ 50): F(50) = 50/10100 = 50 Γ 51 / 10100 = 2550/10100 = 255/1010 = 51/202 β 0.2525
π Verification: At x = 100: F(100) = 100/10100 = 100 Γ 101 / 10100 = 10100/10100 = 1 β
This confirms the CDF is valid since the total probability equals 1 for the maximum value of X.
β Key Takeaways
The lecture demonstrates how to derive a CDF for a discrete random variable when the PMF follows a pattern. The total number of outcomes must first be computed (here, 5050 slips). The PMF is p(i) = i/5050 for i = 1 to 100. The CDF at any x is the sum of probabilities for all values β€ x, which reduces to the sum of the first [x] integers divided by 5050, yielding the closed form F(x) = x/10100. Always verify the CDF equals 1 at the maximum value.
π§ Quick Revision Questions
- What is the total number of slips in the urn in this example?
- What is the probability that the drawn slip shows the number 30?
- What does the notation [x] mean in the CDF formula?
- What is F(75) using the derived formula? Show your work.
- How can you verify that F(x) is a valid cumulative distribution function for this discrete random variable?
π Lecture 15 β Obtaining the PMF from the CDF
π Overview: This lecture teaches how to derive the probability mass function (PMF) from a given cumulative distribution function (CDF). This is a fundamental skill in probability theory, as it allows one to reverse-engineer the underlying discrete probability distribution.
ποΈ Topics Covered
The lecture focuses on a single worked example: given a CDF defined as ( F(x) = \sum_{j=1}^x \left(\frac{1}{2}\right)^j ), find its corresponding PMF. It systematically demonstrates how to compute the probability at each individual point by finding the difference between successive CDF values.
π Lecture Summary
Obtaining the PMF from the CDF
The lecture presents a method for obtaining the PMF from a CDF for discrete random variables. The key principle is that for a discrete random variable, the PMF at a specific point (x) is the difference between the CDF at (x) and the CDF at the point immediately before (x). This is because the CDF accumulates all probability up to and including (x), while the PMF gives the probability exactly at (x).
Given the CDF ( F(x) = \sum_{j=1}^x \left(\frac{1}{2}\right)^j ), the PMF ( f(x) ) is found using the relation ( f(x) = F(x) - F(x-1) ). The lecture then computes the PMF for various values of (x):
- For (x=1): ( f(1) = F(1) - F(0) = \frac{1}{2} - 0 = \frac{1}{2} )
- For (x=2): ( f(2) = F(2) - F(1) = \left(\frac{1}{2} + \frac{1}{4}\right) - \frac{1}{2} = \frac{1}{4} )
- For (x=3): ( f(3) = F(3) - F(2) = \left(\frac{1}{2} + \frac{1}{4} + \frac{1}{8}\right) - \left(\frac{1}{2} + \frac{1}{4}\right) = \frac{1}{8} )
The pattern continues such that the PMF is ( f(x) = \left(\frac{1}{2}\right)^x ). π‘ Why this matters: This pattern reveals that each successive probability is half the previous one, confirming that the CDF defines a geometric distribution with parameter (p = \frac{1}{2}).
π‘ Why this matters: The relationship ( f(x) = F(x) - F(x-1) ) is a cornerstone for working with discrete distributions. It allows you to reconstruct the entire probability distribution from the cumulative function.
π Definition β PMF: The probability mass function of a discrete random variable gives the probability that the variable is exactly equal to a specific value. π Formula: For a discrete random variable (X), the PMF (f(x)) from the CDF (F(x)) is ( f(x) = F(x) - F(x-1) ) β The probability at exactly (x) is the cumulative probability up to (x) minus the cumulative probability up to just before (x). π Example: Given ( F(x) = \sum_{j=1}^x \left(\frac{1}{2}\right)^j ), find ( f(2) ):
- Compute ( F(2) = \frac{1}{2} + \frac{1}{4} = \frac{3}{4} )
- Compute ( F(1) = \frac{1}{2} )
- Then ( f(2) = F(2) - F(1) = \frac{3}{4} - \frac{1}{2} = \frac{1}{4} )
β Key Takeaways
The most critical concept from this lecture is that for discrete random variables, the PMF can be directly obtained from the CDF by taking successive differences: ( f(x) = F(x) - F(x-1) ). This relationship is the inverse of the CDF accumulation process. The example demonstrated that the CDF ( F(x) = \sum_{j=1}^x (1/2)^j ) yields the PMF ( f(x) = (1/2)^x ), which is a geometric distribution. Understanding this one-to-one correspondence between CDF and PMF is essential for analyzing discrete probability distributions.
π§ Quick Revision Questions
- What is the general formula for obtaining the PMF from a CDF for a discrete random variable?
- Given the CDF ( F(x) = \sum_{j=1}^x \left(\frac{1}{2}\right)^j ), what is the value of ( f(4) )?
- Why is it necessary to consider ( F(x-1) ) when finding the PMF at point (x)?
- In the example from this lecture, what probability distribution does the PMF ( f(x) = (1/2)^x ) represent?
- If the CDF of a discrete random variable is ( F(x) = 1 - (1/3)^x ) for (x = 1, 2, 3, ...), what would be the corresponding PMF?
π Lecture 16 β Example of the CDF of a Continuous Random Variable
π Overview: This lecture provides an illustrative example of the Cumulative Distribution Function (CDF) for a continuous random variable. It then introduces the Exponential Distribution, presenting both its Probability Density Function (PDF) and CDF for different mean values (0.5, 1, and 1.5), showing how the distribution changes with the parameter.
ποΈ Topics Covered
This lecture covers an example of the CDF of a continuous random variable where the function is piecewise defined over intervals. It then covers the PDF of the Exponential Distribution for three different mean values of 0.5, 1, and 1.5, followed by the CDF of the Exponential Distribution for the same mean values.
π Lecture Summary
Example of the CDF of a Continuous Random Variable
For a continuous random variable X, the Cumulative Distribution Function (CDF) is given as a piecewise function. Specifically, F(x) = 0 for all x values which are less than zero. F(x) = x for all those x values that lie between 0 and 1. F(x) = 1 for all those x values that are greater than 1.
π Definition β Cumulative Distribution Function (CDF): A function F(x) that gives the probability that a random variable X is less than or equal to a specific value x. π Formula: F(x) = P(X β€ x) β The probability that X is less than or equal to x. π Example: For the given continuous random variable X, to find P(X β€ 0.5): Since 0.5 lies between 0 and 1, F(0.5) = 0.5. Therefore, the probability that X is less than or equal to 0.5 is 0.5.
PDF of the Exponential Distribution with mean=0.5, 1, 1.5
The Probability Density Function (PDF) of the Exponential Distribution is presented for three different mean values. The mean is represented by the parameter. The general form of the PDF is f(x) = (1/mean) * e^(-x/mean) for 0 < x < β. For mean = 0.5, the PDF is f(x) = (1/0.5) * e^(-x/0.5), displayed in orange. For mean = 1, the PDF is f(x) = (1/1) * e^(-x/1) = e^(-x), displayed in purple. For mean = 1.5, the PDF is f(x) = (1/1.5) * e^(-x/1.5), displayed in sky blue. The graph shows how the shape of the exponential distribution varies with different mean values. π‘ Why this matters: The mean parameter directly affects the rate of decay of the exponential distribution.
π Definition β Probability Density Function (PDF): A function f(x) that describes the relative likelihood of a continuous random variable taking on a given value. π Formula: For Exponential Distribution: f(x) = (1/ΞΌ) e^(-x/ΞΌ) for x β₯ 0 β The probability density at point x depends on the mean ΞΌ. π Example: For the Exponential Distribution with mean=1, the PDF is f(x) = e^(-x) for 0 < x < β. This specific distribution is shown in purple.
CDF of the Exponential Distribution with mean=0.5, 1, 1.5
The Cumulative Distribution Function (CDF) of the Exponential Distribution is presented for the same three mean values. The general formula for the CDF of an exponential distribution is F(x) = 1 - e^(-x/mean) for 0 < x < β. For mean = 0.5, the CDF is F(x) = 1 - e^(-x/0.5). For mean = 1, the CDF is F(x) = 1 - e^(-x/1) = 1 - e^(-x). For mean = 1.5, the CDF is F(x) = 1 - e^(-x/1.5). The CDF provides the cumulative probability up to a given value x.
π Definition β CDF of Exponential Distribution: F(x) = 1 - e^(-x/ΞΌ) for x β₯ 0 β The probability that X is less than or equal to x. π Formula: F(x) = 1 - e^(-x/ΞΌ) β Gives P(X β€ x) for an exponential random variable with mean ΞΌ. π Example: For the Exponential Distribution with mean=1, the CDF is F(x) = 1 - e^(-x). To find P(X β€ 1), we compute F(1) = 1 - e^(-1) = 1 - 0.3679 = 0.6321. Therefore, the probability that X is less than or equal to 1 is approximately 0.6321.
β Key Takeaways
The CDF for a continuous random variable can be a piecewise function, ranging from 0 to 1 and increasing as x increases. The Exponential Distribution is defined by its mean parameter, and its PDF has the form (1/mean)*e^(-x/mean). The corresponding CDF of the Exponential Distribution is given by F(x) = 1 - e^(-x/mean), which is used to calculate cumulative probabilities. Different mean values (0.5, 1, 1.5) produce different shapes for both the PDF and CDF, demonstrating how the parameter influences the spread of the distribution. Understanding the relationship between the PDF and CDF is essential for computing probabilities for exponential random variables.
π§ Quick Revision Questions
- What is the value of the CDF, F(x), for a continuous random variable when x is less than 0, according to the example given?
- Write the general formula for the probability density function (PDF) of the Exponential Distribution.
- What is the CDF of the Exponential Distribution with mean = 1.5?
- If an exponential random variable has a mean of 0.5, what is its PDF function?
- Using the CDF formula, calculate P(X β€ 0.5) for an Exponential Distribution with mean = 1.
π Lecture 17 β Concept of two Random Variables being Equal in Distribution
π Overview: This lecture introduces the concept of two random variables being equal in distribution, which is a weaker form of equality than actual identity. It clarifies that two random variables can have the same distribution function even when they are different random variables. An example using the uniform distribution illustrates this important distinction.
ποΈ Topics Covered
Definition of equality in distribution for two random variables; notation for equality in distribution; distinction between equality in distribution and actual equality of random variables; example illustrating that X and Y = 1-X are different but equal in distribution; calculation of CDF for transformed variables; conceptual clarification that equal distribution does not imply identical random variables.
π Lecture Summary
Equality in Distribution β Definition and Concept
We first consider the situation when two random variables X and Y are not equal in distribution: Fβ(x) β Fα΅§(x). We say that X and Y are equal in distribution if and only if Fβ(x) = Fα΅§(x), for all x β β. Mathematically we express it as X β Y (or X = D Y).
π‘ Why this matters: It is important to note that while X and Y may be equal in distribution, they may be quite different as actual random variables.
π Definition β Equality in Distribution: Two random variables X and Y are equal in distribution if and only if their cumulative distribution functions are identical for all real numbers x: Fβ(x) = Fα΅§(x), β x β β.
π Example: Recall that X denotes a real number chosen at random between 0 and 1. We obtain the CDF of X. First, if x < 0, then P(X β€ x) = 0. Next, if x β₯ 1, then P(X β€ x) = 1. Finally, for 0 < x < 1, it follows that P(X β€ x) = x - 0 = x. Hence the CDF of X is:
Fβ(x) =
- 0, if x < 0
- x, if 0 β€ x < 1
- 1, if x β₯ 1
Example with Transformation Y = 1 - X
For instance, in the above example define the random variable Y and transform it as Y = 1 - X. Then Y β X. For example, let X = 0.2 then Y = 1 - 0.2 = 0.8. So, when X is 0.2 then Y is 0.8 which is not the same. Therefore it is interesting to note that the space of Y is the interval (0,1), the same as X.
Computing probabilities: F(0.8) = 1 - P(X < 1 - 0.8) = 1 - P(X < 0.2) = 1 - 0.2 = 0.8. F(0.6) = 1 - P(X < 1 - 0.6) = 1 - P(X < 0.4) = 1 - 0.4 = 0.6.
But we know that for the Uniform distribution defined on (0,1), P(X < 0.2) = 0.2. Therefore, P(X < 1 - y) = 1 - y.
Further, the CDF of Y is 0 for y < 0; 1 for y β₯ 1; and for 0 β€ y < 1, it is:
Fα΅§(y) = P(Y β€ y) = P(1 - X β€ y) = P(X β₯ 1 - y) = 1 - P(X < 1 - y) = 1 - (1 - y) = y
Hence, Fα΅§(y) = y, for 0 < y < 1, which is of exactly the same form as the CDF of X.
π Formula: For Y = 1 - X, where X ~ Uniform(0,1), the CDF of Y is Fα΅§(y) = y for 0 β€ y < 1, which is identical to Fβ(x) = x for 0 β€ x < 1.
Therefore, it is clarified that two random variables may not be the same yet, sometimes they may be equal in distribution: i.e., Y β X, but Y β X.
β Key Takeaways
The most critical concept from this lecture is that equality in distribution means two random variables have identical cumulative distribution functions, but this does NOT mean they are the same random variable. Using the uniform(0,1) example, X and Y = 1-X are clearly different β when X = 0.2, Y = 0.8 β yet their CDFs are identical. This shows that distributional equality is a property of probability laws, not of the random variables themselves. Students must remember the notation X β Y and understand that the CDF completely characterizes distribution. The key distinction between "equal as random variables" and "equal in distribution" is fundamental for advanced probability and statistics.
π§ Quick Revision Questions
- What is the formal mathematical definition of two random variables being equal in distribution?
- Does X β Y imply that X = Y? Explain with an example.
- For X ~ Uniform(0,1) and Y = 1 - X, what is Fα΅§(0.3) and Fα΅§(0.7)?
- Why must the equality Fβ(x) = Fα΅§(x) hold for ALL real x, not just some values?
- If two random variables have the same probability density function, are they necessarily equal in distribution? Are they necessarily the same random variable?
π Lecture 18 β First property of Cumulative Distribution Function and its Proof
π Overview: This lecture introduces the first fundamental property of the Cumulative Distribution Function (CDF) β the property of nondecreasing. Through a formal theorem and proof, it demonstrates that if a random variable has a CDF F(x), then for any two values a and b where a < b, the corresponding probabilities must satisfy F(a) β€ F(b). This property is essential for understanding how cumulative probabilities behave as we move along the real number line.
ποΈ Topics Covered
The lecture covers Theorem 1 concerning the first property of the Cumulative Distribution Function, specifically the nondecreasing property. It presents the formal statement of the theorem along with its complete proof, which relies on the subset relationship between events and the probability monotonicity theorem.
π Lecture Summary
Theorem 1: First Property of Cumulative Distribution Function
Let X be a random variable with cumulative distribution function F(x). Then part (a) states: For all a and b, if a < b, then F(a) β€ F(b). This means F is nondecreasing.
The proof of part (a) relies on the subset relationship between events. Because a < b, we can observe that the interval {X β€ a} is a subset of {X β€ b}. In other words, {X β€ a} β {X β€ b}.
The theorem states: If Cβ and Cβ are events such that Cβ β Cβ, then P(Cβ) β€ P(Cβ).
π‘ Why this matters: This probability monotonicity theorem is the fundamental logical link that transforms the set inclusion relationship into an inequality of probabilities.
Applying this theorem, since the interval X β€ a is contained in the interval X β€ b, therefore the probability of X β€ a is less than or equal to the probability of X β€ b. This gives us F(a) β€ F(b), which directly proves the nondecreasing property.
Example: Consider a random variable X with CDF F(x). Let a = 2 and b = 5, so a < b. The event {X β€ 2} is contained within {X β€ 5} because any value less than or equal to 2 is automatically less than or equal to 5. Therefore, by the theorem, P(X β€ 2) β€ P(X β€ 5), which means F(2) β€ F(5).
β Key Takeaways
The first property of the Cumulative Distribution Function establishes that F(x) is nondecreasing β as the value of x increases, the cumulative probability cannot decrease. This property is proved using the fundamental theorem that if one event is a subset of another event, the probability of the subset event is less than or equal to the probability of the superset event. The proof depends on recognizing that for a < b, the event {X β€ a} is always contained in {X β€ b} because any x satisfying the first inequality automatically satisfies the second. This property is essential for validating that any proposed function can serve as a legitimate CDF.
π§ Quick Revision Questions
- State the first property of the Cumulative Distribution Function as given in Theorem 1.
- What is the set inclusion relationship between {X β€ a} and {X β€ b} when a < b?
- What theorem connects set inclusion to probability inequality in this proof?
- If F(3) = 0.4 and F(7) = 0.4, does this violate the nondecreasing property? Explain.
- Why must the interval X β€ a be a subset of X β€ b when a < b?
π Lecture 19 β Second property of Cumulative Distribution Function and its Proof
π Overview: This lecture focuses on the second property of the cumulative distribution function (CDF), which states that the lower limit of F(x) as x approaches negative infinity is 0. The lecture provides a conceptual explanation and a formal mathematical proof of this essential property.
ποΈ Topics Covered
The lecture covers Theorem 1(b) regarding the CDF, explaining that as x approaches negative infinity, F(x) approaches 0. It provides both a conceptual understanding based on the definition of F(x) as P(X < x) and an impossible event, and a mathematical formulation using limits.
π Lecture Summary
Theorem 1(b): The Lower Limit of F(x) is 0
Let X be a random variable with cumulative distribution function F(x). Then, property (b) states: lim_{xβ-β} F(x) = 0 (the lower limit of F is 0).
We know that F(x) = P(X < x), so obviously F(-β) = P(X < -β), which is an impossible event. Therefore, F(-β) = 0.
Mathematically, we write: lim_{xβ-β} F(x) = 0.
π‘ Why this matters: This property establishes that as we move to the extreme left of the distribution (towards negative infinity), the cumulative probability becomes zero β meaning no probability mass exists below any finite lower bound in a proper probability distribution.
β Key Takeaways
The cumulative distribution function F(x) has the property that its lower limit is zero as x approaches negative infinity. This is because F(x) = P(X < x), and the probability of X being less than negative infinity is an impossible event, which has probability zero. Mathematically, this is expressed as lim_{xβ-β} F(x) = 0. This property is fundamental for understanding the behavior of probability distributions at their lower extremes and ensures that all probability is accumulated as x increases.
π§ Quick Revision Questions
- What does the notation F(x) represent in probability theory?
- What is the second property of the cumulative distribution function discussed in this lecture?
- Why does F(-β) equal 0? Explain in terms of probability theory.
- How is the second property of the CDF expressed mathematically using limits?
- What type of event is represented by P(X < -β), and what is its probability?
π Lecture 20 β Third property of Cumulative Distribution Function and its Proof
π Overview: This lecture presents the third property of the cumulative distribution function (CDF), demonstrating that as x approaches infinity, the CDF approaches 1. This property establishes that the upper limit of any cumulative distribution function is always 1, confirming that the total probability under any probability distribution equals 1.
ποΈ Topics Covered
The lecture covers the third property of the cumulative distribution function F(x), specifically proving that the limit of F(x) as x approaches infinity equals 1. It provides both a mathematical proof using probability concepts and a graphical illustration showing how any CDF starts from level 0 and rises to level 1, after which it continues to infinity.
π Lecture Summary
Third property of Cumulative Distribution Function and its Proof
Theorem 1: Let X be a random variable with cumulative distribution function F(x). Then, (c) lim xββ F(x) = 1 (the upper limit of F is 1).
Proof: We know that F(x) = P(X < x), so obviously F(β) = P(X < β) is a sure event. Therefore, F(β) = 1, hence lim xββ F(x) = 1.
π Definition β Cumulative Distribution Function (CDF): F(x) = P(X < x), the probability that the random variable X takes a value less than x.
π Formula: lim xββ F(x) = 1 β As x increases without bound, the cumulative probability approaches certainty (1), because all possible values of X are less than infinity.
π Example (Graphical Interpretation): The F(x) of any random variable will always start from level 0 and rise up to level 1. Subsequent to that point 1, it goes up to infinity. This means the CDF begins at 0 for very small x values, increases as x increases, and eventually reaches and stays at 1 for sufficiently large x values, indicating all probability has been accumulated.
π‘ Why this matters: This property ensures that the total probability under any probability distribution sums to exactly 1, which is a fundamental requirement for any valid probability distribution.
β Key Takeaways
The third property of the cumulative distribution function states that the limit of F(x) as x approaches infinity equals 1, meaning the upper limit of any CDF is always 1. This is proven logically because F(β) = P(X < β) represents a sure event, and the probability of a sure event is 1. Graphically, every CDF starts at level 0 and rises to level 1, after which it extends to infinity, demonstrating that all probability mass is captured by the time x reaches sufficiently large values. Students must remember that this property, along with the first two properties (F(x) is non-decreasing and lim xβ-β F(x) = 0), completes the essential characteristics of any valid cumulative distribution function.
π§ Quick Revision Questions
- What is the third property of the cumulative distribution function?
- Why does F(β) equal 1?
- What does the graphical representation of a CDF look like regarding its starting and ending levels?
- What is the relationship between F(β) and the concept of a sure event?
- How does the third property of CDF relate to the total probability of a random variable?
π Lecture 21 β Fourth property of Cumulative Distribution Function and its Proof
π Overview: This lecture completes the discussion of the four properties of the Cumulative Distribution Function (CDF) by proving the fourth property β right continuity. Understanding why CDFs are right continuous is essential for working with probability distributions, especially when dealing with discrete random variables and points of discontinuity.
ποΈ Topics Covered
The lecture presents Theorem 1 part (d) regarding the right continuity of the Cumulative Distribution Function. It provides an explanation using the Discrete Uniform distribution as an example with a fair die toss, illustrates the concept through a graphical representation of a continuous CDF approaching a discontinuity point from the right, and presents the formal proof using a decreasing sequence of sets and the continuity theorem for probability measures.
π Lecture Summary
Fourth property of Cumulative Distribution Function and its Proof
Theorem 1: Let X be a random variable with cumulative distribution function F(x). Then (d) lim_(xβxβ) F(x) = F(xβ) (F is right continuous).
Explanation: In case of a discrete distribution, we take an example of a Discrete Uniform distribution.
- Toss a fair die (1,2,3,4,5,6).
- The probability of getting a 1 or 2 or 3 or 4 or 5 or 6 are equally likely to occur.
The table shows:
| x | P(x) | F(x) |
|---|---|---|
| 1 | 1/6 | 1/6 |
| 2 | 1/6 | 1/6+1/6=2/6 |
| 3 | 1/6 | 2/6+1/6=3/6 |
| 4 | 1/6 | 3/6+1/6=4/6 |
| 5 | 1/6 | 4/6+1/6=5/6 |
| 6 | 1/6 | 5/6+1/6=1 |
Consider a CDF that is continuous everywhere other than at the point xβ (ref. CDF's that have a few points of discontinuity). As x tends to xβ from the R.H.S (right-hand side), the ordinate F(x) tends to the ordinate F(xβ).
π‘ Why this matters: The right continuity property means that at any point where the CDF has a jump, the function value equals the limit from the right. This distinguishes CDFs from other cumulative functions and has important implications for how probabilities are assigned at individual points.
Part (d): Formal Proof
Let {xβ} be any sequence of real numbers such that xβ β xβ. Let Cβ = {X β€ xβ}. Then the sequence of sets {Cβ} is decreasing and
ββββ^β Cβ = {X β€ xβ}
π Definition β Right Continuous: A function F is right continuous at a point xβ if the limit of F(x) as x approaches xβ from values greater than xβ equals F(xβ). That is, lim_(xβxβ) F(x) = F(xβ).
π Formula: lim_(xβxβ) F(x) = F(xβ) β As we approach xβ from the right side, the value of the CDF approaches the actual value of the CDF at xβ.
π Example: Using the fair die distribution, consider xβ = 1. The limit as x approaches 1 from the right is F(1) = 1/6. For the discrete uniform distribution, F(1) = P(X β€ 1) = P(X = 1) = 1/6. Approaching from the right, we never take values below 1, so the limit equals exactly 1/6.
Hence, by Theorem 3.6 (which states that for a decreasing sequence of sets, the probability of the intersection equals the limit of the probabilities), we have:
lim_(nββ) F(xβ) = F(xβ)
which is the desired result that the CDF is right continuous.
β Key Takeaways
The fourth fundamental property of a Cumulative Distribution Function is right continuity, meaning that at any point xβ, the limit of F(x) as x approaches xβ from the right equals F(xβ). The proof uses a decreasing sequence of sets {X β€ xβ} whose intersection is {X β€ xβ} and applies the continuity theorem for probability measures. For discrete distributions like the uniform die toss, right continuity holds even at points of discontinuity, meaning the CDF takes the value at the point itself rather than the limit from the left. The formal proof requires considering any decreasing sequence converging to xβ from the right and showing the corresponding set probabilities converge. This completes the set of four properties that characterize any valid cumulative distribution function.
π§ Quick Revision Questions
- State the fourth property of the Cumulative Distribution Function as given in Theorem 1 part (d).
- In the proof of right continuity, what type of sequence {xβ} is considered, and what does xβ β xβ mean?
- What is the relationship between the sequence of sets Cβ = {X β€ xβ} and the set {X β€ xβ} as nββ?
- Which theorem from probability theory is used to complete the proof of right continuity?
- For a fair die toss, what is the value of F(2) and why does right continuity hold at x=2?
π Lecture 22 β Evaluating Probabilities using CDF
π Overview: This lecture introduces the fundamental theorem for computing probabilities using the Cumulative Distribution Function (CDF). It demonstrates how the CDF can be used to find probabilities for intervals of a random variable (X), replacing the need to work directly with probability density functions in many cases.
ποΈ Topics Covered
The lecture covers Theorem 1, which states that for a random variable (X) with CDF (F_X(x)), the probability (P[a < X < b]) equals (F_X(b) - F_X(a)). It provides a formal proof using the addition theorem of probability on mutually exclusive intervals. Finally, a numerical example is worked out using the exponential distribution to illustrate the application of the theorem.
π Lecture Summary
Theorem 1: Evaluating (P[a < X < b]) using CDF
Let (X) be a random variable with the CDF (F_X(x)). Then, for (a < b): [ P[a < X < b] = F_X(b) - F_X(a) ]
Proof: The interval ({-\infty < X < b}) can be written as the union of two mutually exclusive intervals: [ {-\infty < X < b} = {-\infty < X < a} \cup {a < X < b} ] Since these two intervals are mutually exclusive (they do not overlap), we apply the addition theorem of probability, which states that the probability of the union of mutually exclusive events equals the sum of their probabilities: [ P{-\infty < X < b} = P{-\infty < X < a} + P{a < X < b} ] Rearranging the terms yields: [ P{a < X < b} = P{-\infty < X < b} - P{-\infty < X < a} ] By definition of the CDF, (F_X(x) = P{-\infty < X < x}). Therefore: [ P[a < X < b] = F_X(b) - F_X(a) ]
π‘ Why this matters: This theorem simplifies probability calculations dramatically. Instead of integrating a probability density function over an interval, you can simply take the difference of two CDF values β a much faster and more practical approach.
π Definition β Mutually Exclusive Events: Two events are mutually exclusive if they cannot occur simultaneously. Their intersection is empty, so the probability of their union is simply the sum of their individual probabilities.
π Formula: (P[a < X < b] = F_X(b) - F_X(a)) β The probability that a random variable (X) falls in the interval ((a, b)) equals the CDF evaluated at (b) minus the CDF evaluated at (a).
Numerical Example
Find (P(1 < X < 2)) where (X) follows an exponential distribution with parameter (\theta = 2).
Step 1: Recall the CDF of an exponential distribution with parameter (\theta): [ F(x) = 1 - e^{-x/\theta}, \quad x \geq 0 ] Here (\theta = 2), so: [ F(x) = 1 - e^{-x/2} ]
Step 2: Apply the theorem: [ P(1 < X < 2) = F(2) - F(1) ]
Step 3: Calculate (F(2)): [ F(2) = 1 - e^{-2/2} = 1 - e^{-1} = 1 - \frac{1}{e} ]
Step 4: Calculate (F(1)): [ F(1) = 1 - e^{-1/2} = 1 - \frac{1}{\sqrt{e}} ]
Step 5: Compute the difference: [ P(1 < X < 2) = (1 - e^{-1}) - (1 - e^{-1/2}) = e^{-1/2} - e^{-1} ]
Step 6: Evaluate numerically using (e \approx 2.71828):
- (e^{-1/2} = \frac{1}{\sqrt{e}} = \frac{1}{\sqrt{2.71828}} = \frac{1}{1.64872} = 0.6065)
- (e^{-1} = \frac{1}{e} = \frac{1}{2.71828} = 0.3679)
Step 7: Final answer: [ P(1 < X < 2) = 0.6065 - 0.3679 = 0.2386 = 23.86% ]
π Example: For an exponential random variable with (\theta = 2), the probability that (X) lies between 1 and 2 is approximately 23.86%. This is found by computing (F(2) - F(1) = e^{-1/2} - e^{-1} \approx 0.2386).
β Key Takeaways
The most critical concepts from this lecture are: (1) The probability that a continuous random variable (X) falls in any interval ((a, b)) can be computed directly from its CDF using the formula (P[a < X < b] = F_X(b) - F_X(a)). (2) This theorem is proved by expressing the interval as a union of mutually exclusive events and applying the addition theorem of probability. (3) The exponential distribution CDF, (F(x) = 1 - e^{-x/\theta}), was used in the example to demonstrate the method. (4) Numerical evaluation requires careful substitution and use of the exponential function. (5) This approach is much simpler than integrating the PDF, making CDFs a powerful tool for probability calculations.
π§ Quick Revision Questions
- State Theorem 1 for evaluating probabilities using the CDF.
- What is the key property of the two intervals that allows the addition theorem to be applied in the proof?
- In the example, what is the formula for the exponential CDF when (\theta = 2)?
- What is the numerical value (to four decimal places) of (e^{-1/2}) and (e^{-1})?
- If (F_X(3) = 0.8) and (F_X(1) = 0.2), what is (P(1 < X < 3))?