STA642 — Midterm Summary (Lectures 1–22)

📘 Lecture 2 — Definition of Random Variable

📖 Overview: This lecture defines the concept of a random variable as a function that maps outcomes of a random experiment to real numbers. It distinguishes between discrete and continuous random variables, which is fundamental for understanding probability distributions and statistical inference.

🗂️ Topics Covered

The lecture covers the concept of a random variable as a real-valued function defined on a sample space, with an example of a coin toss where heads maps to 1 and tails maps to 0. It then formally defines a random variable and its range or space, explains how a random variable induces a probability distribution, and distinguishes between discrete random variables (finite or countably infinite range) and continuous random variables (interval range).

📝 Lecture Summary

Concept of a Random Variable

A random variable is a variable that is associated with a random experiment. For example, consider the random experiment of tossing a fair coin. The possible outcomes are Head and Tail, and the sample space is ( C = {H, T} ).

Let ( X ) be a function such that ( X(T) = 0 ) and ( X(H) = 1 ). Thus, ( X ) is a real-valued function defined on the sample space ( C ). ( X ) takes us from the sample space ( C ) (domain = ( {H, T} )) to a space of real numbers ( D = {0, 1} ) (range).

💡 Why this matters: This mapping from outcomes to numbers is the foundation for applying mathematical probability to real-world experiments.

Definition

Consider a random experiment with a sample space ( C ). A function ( X ), which assigns to each element ( c \in C ) one and only one number ( X(c) = x ), is called a random variable. The space or range of ( X ) is the set of real numbers ( D = { x : x = X(c), c \in C } ).

Given a random variable ( X ), its range ( D ) becomes the space of interest. Besides inducing the space ( D ), ( X ) also induces a probability which we call the distribution of ( X ). For the fair coin example, the probability of getting a 0 is ( 1/2 ), and the probability of getting a 1 is also ( 1/2 ).

🔑 Definition — Random Variable: A function ( X ) that assigns to each element ( c ) in the sample space ( C ) exactly one real number ( X(c) = x ). 📐 Notation: ( X: C \rightarrow D ), where ( D = { x : x = X(c), c \in C } ) → ( X ) maps outcomes to numbers. 📌 Example: Coin toss: ( X(H) = 1 ), ( X(T) = 0 ). Domain = ( {H, T} ), Range = ( {0, 1} ). Probability: ( P(X=0) = 1/2 ), ( P(X=1) = 1/2 ).

Discrete vs. Continuous Random Variables

The set ( D ) can be a finite set, a countable set, or an interval of real numbers.

• When ( D ) is finite or countably infinite, the random variable is called a discrete random variable.
• When ( D ) is an interval, the random variable is called a continuous random variable.

Discrete random variable has two situations:

1. ( D ) is a finite set, e.g., coin toss: ( D = {0, 1} ), or fair die toss: ( D = {1, 2, 3, 4, 5, 6} ).
2. ( D ) is a countably infinite set, e.g., ( D = {1, 2, 3, \ldots} ).

In both cases, the elements may be called ( d_1, d_2, \ldots, d_m ) (if finite) or ( d_1, d_2, \ldots ) (if countably infinite).

🔑 Definition — Discrete Random Variable: A random variable whose range ( D ) is a finite set or a countably infinite set. 🔑 Definition — Continuous Random Variable: A random variable whose range ( D ) is an interval of real numbers. 📌 Example: Finite discrete: Tossing a fair die gives ( D = {1, 2, 3, 4, 5, 6} ). Countably infinite discrete: The number of tosses until the first head appears gives ( D = {1, 2, 3, \ldots} ).

⭐ Key Takeaways

A random variable is a function that maps each outcome of a random experiment to a real number, creating a bridge between sample spaces and numerical analysis. The range of a random variable becomes the space of interest, and the random variable induces a probability distribution over this range. Random variables are classified as discrete (finite or countably infinite range) or continuous (interval range). For a fair coin toss, the random variable mapping heads to 1 and tails to 0 assigns equal probabilities of 1/2 to each value. Understanding this classification is essential for choosing the correct probability model in statistical problems.

🧠 Quick Revision Questions

1. What is a random variable, and how is it defined in terms of a sample space?
2. In the coin toss example, what are the domain and range of the random variable ( X )?
3. What is the probability distribution induced by ( X ) for a fair coin?
4. What are the two types of random variables, and how are they distinguished?
5. Give an example of a discrete random variable with a finite range and one with a countably infinite range.

📘 Lecture 3 — Example of Random Variable

📖 Overview: This lecture demonstrates how to construct a probability distribution from a random variable using a concrete example of selecting a card from a standard deck. It shows how to map outcomes to numerical values and compute corresponding probabilities to form a complete probability distribution.

🗂️ Topics Covered

This lecture covers the construction of a discrete probability distribution from a random variable defined on card outcomes. It explains mapping face cards and aces to specific numeric values, computing probabilities using the classical definition, and verifying that probabilities sum to 1.

📝 Lecture Summary

Example: Selecting a Card from a Deck

Consider selecting a card from an ordinary deck of 52 playing cards. Each outcome (card) is assigned a probability of 1/52. We define a random variable X that assigns numeric values based on the card's rank:

• ( X(c) = 4 ) if card ( c ) is an ace
• ( X(c) = 3 ) if card ( c ) is a king
• ( X(c) = 2 ) if card ( c ) is a queen
• ( X(c) = 1 ) if card ( c ) is a jack
• ( X(c) = 0 ) otherwise (for all other cards)

Now, we construct a probability distribution table with two columns. The first column lists the values of the random variable ( X ): 0, 1, 2, 3, 4. The second column lists the corresponding probabilities.

Using the classical definition of probability — favorable outcomes divided by total outcomes — we compute each probability:

• For ( X = 0 ) (any card that is not ace, king, queen, or jack): There are 52 total cards minus 4 aces, 4 kings, 4 queens, and 4 jacks = 36 cards. Probability = 36/52.
• For ( X = 1 ) (jack): 4 jacks in the deck. Probability = 4/52.
• For ( X = 2 ) (queen): 4 queens. Probability = 4/52.
• For ( X = 3 ) (king): 4 kings. Probability = 4/52.
• For ( X = 4 ) (ace): 4 aces. Probability = 4/52.

The sum of all probabilities must equal 1:
( 36/52 + 4/52 + 4/52 + 4/52 + 4/52 = 52/52 = 1 ).

🔑 Definition — Random Variable: A function that assigns a numerical value to each outcome of a random experiment.

📐 Formula (Classical Probability):
( P(E) = \frac{\text{Number of favorable outcomes}}{\text{Total number of possible outcomes}} )

📌 Example:
Compute the probability distribution for the random variable X defined on a single card draw from a standard 52-card deck.
The values of X are 0 (non-face, non-ace card), 1 (jack), 2 (queen), 3 (king), 4 (ace).
Number of cards for each value:

• X=0: 36 cards (all cards except aces, kings, queens, jacks)
• X=1: 4 jacks
• X=2: 4 queens
• X=3: 4 kings
• X=4: 4 aces
  Probabilities: 36/52, 4/52, 4/52, 4/52, 4/52.
  Sum = (36+4+4+4+4)/52 = 52/52 = 1.

💡 Why this matters: Verifying that probabilities sum to 1 confirms that we have a valid probability distribution, which is the foundation for calculating expected values and variances in later topics.

⭐ Key Takeaways

A random variable maps outcomes to numbers, and its probability distribution lists each possible value with its probability. The classical probability formula (favorable/total) is used when outcomes are equally likely. For a deck of cards, each card has probability 1/52, so the probability of a specific rank depends on how many cards have that rank. Always verify that the sum of all probabilities equals 1 to ensure the distribution is valid. This example generalizes to any discrete random variable with a finite number of outcomes.

🧠 Quick Revision Questions

1. What are the five possible values of the random variable X in this example, and what do they represent?
2. How many cards correspond to X = 0, and why?
3. What is the probability that a randomly selected card is a king or an ace?
4. Show that the sum of all probabilities in this distribution equals 1.
5. Using the classical definition, how would the probabilities change if we selected two cards without replacement?

📘 Lecture 4 — Discrete Random Variable

📖 Overview: This lecture defines discrete random variables and distinguishes between finite and countably infinite sample spaces. It introduces the geometric distribution through the example of coin tosses, demonstrating how to compute probabilities for a countably infinite set of outcomes and verify that the sum of probabilities equals one.

🗂️ Topics Covered

The lecture begins by defining a discrete random variable and distinguishing between finite and countably infinite spaces. It provides an example of a finite space using coin flips. It then explores a countably infinite example: the number of tosses needed to obtain the first head, deriving the probability mass function and verifying the sum of probabilities using a geometric series.

📝 Lecture Summary

Definition (Discrete Random Variable)

We say a random variable is a discrete random variable if its space is either finite or countable.

🔑 Definition — Discrete Random Variable: A random variable whose possible values form a finite or countably infinite set.

Example: Finite Space

Consider a sequence of independent flips of a coin, each resulting in a head (H) or a tail (T). Moreover, on each flip, we assume that H and T are equally likely; that is, P(H)=P(T)=1/2. This is the case when the space is Finite.

Let the random variable X equal the number of heads in one flip. The space is D = {0, 1}. The probability distribution is:

• P(X=0) = 1/2
• P(X=1) = 1/2
• Sum = 1

📌 Example: For a single fair coin flip, P(X=0) = 1/2 and P(X=1) = 1/2.

Example of a Countably Infinite Set

Let the random variable X equal the number of tosses needed to obtain the first head. The space of X is D = {1, 2, 3, 4, ...}.

Consider that X=1 when the sequence begins with a H. Likewise, X=2 when the sequence begins with TH. In the same way when X=3 the sequence is TTH.

Suppose that we are interested in finding the probabilities of the various possible values of X. As shown in the table, X is the total number of tosses that was required in order to get the first Head.

The probability mass function is:

📐 Formula: P(X = x) = (1/2)^(x-1) * (1/2) = (1/2)^x, for x = 1, 2, 3, ...

Where X is the total number of tosses required to get the first Head.

Now, let's verify the sum of probabilities:

• P(X=1) = (1/2)^1 = 1/2
• P(X=2) = (1/2)^2 = 1/4
• P(X=3) = (1/2)^3 = 1/8
• ... and so on

Although it is an infinite sequence of probabilities, the sum has to be 1.

The sum is: ∑_(x=1)^∞ (1/2)^x = (1/2) + (1/4) + (1/8) + ...

This is a geometric series with first term a = 1/2 and common ratio r = 1/2.

📐 Formula — Geometric Series: For |r| < 1, the sum of an infinite geometric series is: a + ar + ar² + ... = a / (1 - r)

Substituting a = 1/2 and r = 1/2: Sum = (1/2) / (1 - 1/2) = (1/2) / (1/2) = 1

✅ The sum of probabilities equals 1, confirming that this is a valid probability distribution.

💡 Why this matters: This verifies that even for countably infinite spaces, the total probability must still sum to 1, and the geometric series provides the mathematical tool to confirm this.

⭐ Key Takeaways

A discrete random variable has a finite or countably infinite set of possible values. For the coin-tossing experiment with an infinite sample space, the probability that the first head occurs on the x-th toss follows a geometric distribution: P(X=x) = (1/2)^x. The sum of all probabilities in this infinite geometric series is 1, which is verified using the geometric series formula a/(1-r). The key distinction is between finite spaces (like a single coin flip) and countably infinite spaces (like the number of tosses until first head), both of which qualify as discrete random variables.

🧠 Quick Revision Questions

1. What is the definition of a discrete random variable?
2. In the finite coin flip example, what are the possible values of X and their probabilities?
3. In the countably infinite example, what does the random variable X represent?
4. What is the probability that the first head occurs on the 4th toss?
5. How do you verify that the sum of probabilities for a geometric distribution equals 1?

📘 Lecture 5 — Concept of Probability Mass Function

📖 Overview: This lecture introduces the probability mass function (pmf), a fundamental concept for discrete random variables. It explains how probabilities are assigned to each possible outcome of a random variable and how the cumulative distribution function (CDF) is derived from the pmf, which is essential for understanding probability distributions.

🗂️ Topics Covered

The lecture covers the definition and properties of the probability mass function (pmf), including its two key properties: non-negativity and summation to one. It also introduces the cumulative distribution function (CDF) for discrete random variables, demonstrating how the CDF accumulates probabilities and how the "jump" height in the CDF corresponds to the pmf value at each point.

📝 Lecture Summary

What is Probability Mass Function?

Consider a random experiment with a sample space that is finite or countably infinite. A function X, which assigns to each element c ∈ C one and only one number X(c) = x, is called a random variable. If we express the probabilities against each random value of X algebraically, then we can say that this is the probability mass function of X.

Example: Toss a fair die, the possible values are 1,2,3,4,5,6. Algebraically: P(X = x) = 1/6, for x = 1, 2, 3, 4, 5, 6.

Definition

The probability mass function (pmf) of X is given by: 🔑 Definition — pX(x) = P(X = x), for x ∈ D.

The PMFs satisfy two properties:

1. 0 ≤ pX(x) ≤ 1, for x ∈ D
2. Σ pX(x) = 1, for all x ∈ D

CDF of a Discrete Random Variable

Example: Suppose we roll a fair die with the numbers 1 through 6 on it. Let X be the upface of the roll. Then the space of X is {1,2,....,6} and its pmf is pX(i) = 1/6, for i = 1, 2, ..., 6.

The cumulative distribution function (CDF) is calculated as: F(4) = P(X ≤ 4) = P(X=1 or 2 or 3 or 4) = 1/6 + 1/6 + 1/6 + 1/6 = 4/6

The table below shows the pmf and CDF values:

📐 Formula: F(x) = P(X ≤ x) = Σ p(t) for all t ≤ x → The CDF accumulates all probabilities up to and including the value x.

📌 Example: For the fair die, F(4) = 4/6, meaning the probability of rolling a 4 or less is 4/6.

The height of each 'Jump' in the CDF yields the probability of that particular value of X.

⭐ Key Takeaways

A random variable assigns a unique number to each outcome in the sample space. The probability mass function (pmf) gives the probability of each possible value of a discrete random variable and must satisfy two conditions: each probability is between 0 and 1, and the sum of all probabilities equals 1. The cumulative distribution function (CDF) accumulates probabilities from the smallest value up to a given point, and each "jump" in the CDF corresponds exactly to the pmf value at that point. Understanding the relationship between pmf and CDF is crucial for analyzing discrete probability distributions.

🧠 Quick Revision Questions

1. What are the two conditions that a probability mass function must satisfy?
2. How do you calculate the cumulative distribution function F(x) from the pmf p(x)?
3. For a fair die, what is the pmf value for X = 3?
4. If F(4) = 4/6 for a fair die, what does this probability represent in words?
5. What does the height of a "jump" in the CDF represent?

📘 Lecture 6 — Example of Probability Mass Function

📖 Overview: This lecture demonstrates the application of the probability mass function (pmf) through a concrete example involving drawing slips from an urn. It shows how to derive the pmf mathematically using the classical definition of probability and then compute cumulative probabilities, making the abstract concept tangible and directly applicable to exam problems.

🗂️ Topics Covered

The lecture presents a detailed example of a probability mass function for a random variable X representing the number on a slip drawn from an urn containing slips numbered 1 to 100. It covers the total number of slips using the sum of natural numbers formula, the classical probability definition, deriving the pmf p(x) = x/5050, and computing P(X<50) using the sum formula.

📝 Lecture Summary

Example: Urn with Slips Numbered 1 to 100

Consider an urn containing slips of paper, each with one of the numbers 1, 2, ..., 100 on it. There are i slips with the number i on it for i = 1, 2, ..., 100. For example, there are 25 slips of paper with the number 25. The slips are identical except for the numbers. One slip is drawn at random. Let X be the number on the slip.

a) Suppose that X has the pmf p(x) = x/5050, x = 1,2,3,...,100, zero elsewhere

Solution: Total number of slips = 1 + 2 + 3 + ... + 100.

Using the formula for the sum of the first n natural numbers:

• The sum of the first n natural numbers = n(n+1)/2
• Here, total number of slips = 100(101)/2 = 5050

By the classical definition of probability: Probability = m/n, where m = number of favorable outcomes and n = total number of outcomes.

Since the slips are identical (i.e., equally likely to be drawn), the probability that X = x is: P(X = x) = x/5050, for x = 1, 2, 3, ..., 100

This is the probability mass function (pmf) of X.

🔑 Definition — Probability Mass Function (pmf): A function that gives the probability that a discrete random variable is exactly equal to some value. For a valid pmf, all probabilities must be between 0 and 1, and the sum of all probabilities must equal 1.

📐 Formula: P(X = x) = x/5050, x = 1, 2, ..., 100 → The probability of drawing a slip with number x is equal to x divided by the total number of slips (5050).

b) Compute P(X < 50)

Solution: P(X < 50) = P(X = 1) + P(X = 2) + P(X = 3) + ... + P(X = 49) = (1 + 2 + 3 + ... + 49) / 5050

Using the sum of natural numbers formula for the first 49 numbers: Sum = 49 × 50 / 2 = 1225

Therefore: P(X < 50) = 1225 / 5050 = 49/202 = 0.2426

💡 Why this matters: This example demonstrates the direct link between counting favorable outcomes and the resulting probability mass function. The pmf is not arbitrary — it arises naturally from the problem's structure. Understanding this connection is crucial for constructing pmfs in other contexts.

⭐ Key Takeaways

The most critical concepts from this lecture are: the classical probability definition (m/n) requires identifying both favorable and total outcomes; the total number of slips is obtained using the sum of the first n natural numbers formula n(n+1)/2; the pmf p(x) = x/5050 directly reflects the number of slips with each number; cumulative probabilities like P(X<50) are computed by summing individual pmf values; and the probability P(X<50) = 49/202 = 0.2426 is obtained by applying the sum formula to the numerator.

🧠 Quick Revision Questions

1. What is the total number of slips in the urn, and how is it computed?
2. Why is p(x) = x/5050 a valid probability mass function?
3. Explain the classical definition of probability used in this example.
4. Compute P(X ≤ 30) step-by-step using the formulas from this lecture.
5. If the number of slips with number i was 2i instead of i, how would the pmf change?

📘 Lecture 7 — Continuous Random Variable

📖 Overview: This lecture introduces the concept of a continuous random variable, where probability is defined over intervals rather than at individual points. It covers the cumulative distribution function (CDF) and probability density function (PDF), their properties, and how to compute probabilities through integration.

🗂️ Topics Covered

The lecture begins with the fundamental concept that a continuous random variable can assume any value in an interval, illustrated through height measurement. It then formally defines the cumulative distribution function and probability density function, establishes their relationship via the Fundamental Theorem of Calculus, states the two key properties of PDFs, and explains how to compute probabilities by integrating the PDF. An example of the uniform distribution on (0,1) is provided to demonstrate these concepts.

📝 Lecture Summary

Concept of a Continuous Random Variable

In the continuous case, we cannot define probability on one particular point; instead, probability is defined on an interval. The lecture illustrates this with a height measurement example: if we say a person is 5 feet 4 inches (64 inches) tall, this is a crude measurement. With a more refined instrument, the actual height might be 64.2 inches, which is really somewhere between 64.15 and 64.25 inches. Theoretically, there can be an infinite number of decimal places — this is the basic concept that a continuous variable can assume any value.

Cumulative Distribution Function F(x)

The cumulative distribution function (CDF) of a continuous random variable X is defined as:

🔑 Definition — Cumulative Distribution Function (CDF): The CDF of a continuous random variable X is given by:

📐 Formula: F_X(x) = ∫_{-∞}^{x} f_X(t) dt

Where f_X(t) is called a probability density function (pdf) of X. If f_X(x) is also continuous, then the Fundamental Theorem of Calculus implies:

📐 Formula: d/dx [F_X(x)] = f_X(x)

💡 Why this matters: This relationship allows us to move between the CDF (which gives cumulative probabilities) and the PDF (which gives the density of probability at each point).

Properties of Probability Density Functions

The PDFs satisfy two properties:

1️⃣ ∫_{-∞}^{∞} f_X(x) dx = 1

2️⃣ f_X(x) ≥ 0

🔑 Definition — Probability: In the continuous case, the area under the curve of the PDF gives you the probability.

To compute probabilities, they can be obtained by integration:

📐 Formula: P(a < X ≤ b) = ∫_{a}^{b} f_X(t) dt = F_X(b) - F_X(a)

Example: The Uniform Distribution

Suppose we randomly select a number between 0 and 1. Recall that the CDF of X is:

F_X(x) = x, for 0 < x < 1

Hence the PDF of X is given by:

📐 Formula: f_X(x) = { 1, for 0 < x < 1; 0, elsewhere }

This is called a Uniform distribution or rectangular distribution where 0 < x < 1.

📌 Example: For this uniform distribution on (0,1):

• The PDF is constant (equal to 1) over the interval (0,1) and zero elsewhere
• The CDF is F(x) = x for 0 < x < 1
• The area under the PDF curve over (0,1) equals 1 (satisfying property 1)
• To find P(0.2 < X < 0.7) = ∫_{0.2}^{0.7} 1 dx = 0.5 = F(0.7) - F(0.2)

⭐ Key Takeaways

The most critical concepts from this lecture are: (1) For continuous random variables, probability is defined over intervals, not at individual points. (2) The PDF f_X(x) must be non-negative and integrate to 1 over its entire range. (3) The CDF F_X(x) is obtained by integrating the PDF from negative infinity to x, and conversely, the PDF is the derivative of the CDF. (4) Probabilities for intervals are found by integrating the PDF over that interval or by subtracting CDF values. (5) The uniform distribution on (0,1) is the simplest continuous distribution, with constant PDF equal to 1 over the interval.

🧠 Quick Revision Questions

1. Why can't we define probability at a single point for a continuous random variable?
2. What are the two essential properties that every probability density function must satisfy?
3. If the CDF of a continuous random variable X is F(x) = x² for 0 < x < 1, what is the PDF?
4. How do you compute P(a < X ≤ b) using both the PDF and the CDF?
5. For a uniform distribution on (0,1), what is the probability that X lies between 0.3 and 0.8?

📘 Lecture 8 — Example of Probability Density Function

📖 Overview: This lecture provides a concrete example of a probability density function (pdf) for a continuous random variable. It demonstrates how to calculate probabilities for continuous distributions using integration of the pdf, specifically for the simple experiment of choosing a real number at random from the interval (0,1).

🗂️ Topics Covered

This lecture covers the example of a continuous random variable X representing a number chosen uniformly at random from (0,1). It defines the probability density function as ( f(x) = 1 ) for ( 0 < x < 1 ), and calculates the probability that X is less than an eighth or greater than seven eighths using integration of the pdf.

📝 Lecture Summary

Example: Choosing a Number at Random from (0,1)

In the continuous case, consider the following simple experiment: choose a real number at random from the interval (0,1). Let X be the number chosen. In this case the space of X is D = (0,1).

Since the number is chosen at random, it is reasonable to assign: [ P(a < X < b) = b - a \quad \text{for} \quad 0 < a < b < 1 ]

🔑 Definition — Probability Density Function (pdf): For a continuous random variable, the function ( f(x) ) such that the probability that X lies in an interval [a, b] is given by the integral of ( f(x) ) from a to b.

It follows that the pdf of X is: [ f(x) = \begin{cases} 1 & \text{for } 0 < x < 1 \ 0 & \text{elsewhere} \end{cases} ]

📐 Formula: For this uniform distribution on (0,1): ( f(x) = 1 ) for ( 0 < x < 1 ), zero elsewhere → The probability is the length of the interval.

📌 Example: Find the probability that X is less than an eighth or greater than seven eighths.

[ P\left(X < \frac{1}{8} \cup X > \frac{7}{8}\right) = \int_{0}^{\frac{1}{8}} 1 \cdot dx + \int_{\frac{7}{8}}^{1} 1 \cdot dx ]

[ = \left[ x \right]{0}^{\frac{1}{8}} + \left[ x \right]{\frac{7}{8}}^{1} ]

[ = \left( \frac{1}{8} - 0 \right) + \left( 1 - \frac{7}{8} \right) ]

[ = \frac{1}{8} + \frac{1}{8} = \frac{2}{8} = \frac{1}{4} ]

[ \therefore P\left(X < \frac{1}{8} \cup X > \frac{7}{8}\right) = \frac{1}{4} ]

💡 Why this matters: This example shows how continuous probability calculations replace summation (used in discrete cases) with integration of the pdf.

⭐ Key Takeaways

The pdf for a uniform distribution on (0,1) is ( f(x) = 1 ) for ( 0 < x < 1 ). Probability calculations for continuous random variables involve integrating the pdf over the desired interval. For this uniform case, probability equals the length of the interval. The probability that X is less than 1/8 or greater than 7/8 is 1/4. This example demonstrates the fundamental difference between discrete and continuous probability: continuous probabilities require integration rather than summation.

🧠 Quick Revision Questions

1. What is the probability density function for a random number chosen from the interval (0,1)?
2. How do you calculate the probability that X is less than 1/8 for this continuous uniform distribution?
3. For this distribution, what does ( P(a < X < b) ) equal in simple terms?
4. What is the method for calculating ( P(X < 1/8 \cup X > 7/8) )?
5. What is the final probability that X is less than an eighth or greater than seven eighths?

📘 Lecture 9 — Probability Generating Function

📖 Overview: This lecture introduces the probability generating function (PGF) as a power series representation of the probability mass function of a discrete random variable. It provides a powerful tool for deriving moments and analyzing distributions, with examples for the Bernoulli and Poisson distributions.

🗂️ Topics Covered

Definition of the probability generating function in the univariate case, derivation of PGF for Bernoulli random variable with parameter p, derivation of PGF for Poisson distribution with parameter λ, and application of exponential series expansion.

📝 Lecture Summary

Probability Generating Function

In probability theory, the probability generating function of a discrete random variable is a power series representation (the generating function) of the probability mass function of the random variable.

Definition — Univariate Case

If X is a discrete random variable taking values in the nonnegative integers {0, 1, ...}, then the probability generating function of X is defined as:

🔑 Definition — Probability Generating Function (PGF): ( G_X(z) = E[z^X] = \sum_{x=0}^{\infty} p(x) z^x ), where p(x) is the probability mass function of X.

📌 Example: Bernoulli Random Variable The probability generating function of a Bernoulli random variable with parameter p, where X represents the number of heads obtained when tossing a fair coin once, is given by: [ G(z) = \frac{1}{2} + \frac{1}{2}z ]

Poisson Distribution

For the Poisson Distribution with parameter λ:

📐 Formula: ( G(z) = e^{\lambda(z - 1)} )

The derivation proceeds as follows:

Starting with the Poisson PMF: ( p(x) = \frac{e^{-\lambda} \lambda^x}{x!} )

The PGF is: [ G(z) = \sum_{x=0}^{\infty} p(x) z^x = \sum_{x=0}^{\infty} \frac{e^{-\lambda} \lambda^x}{x!} z^x = e^{-\lambda} \sum_{x=0}^{\infty} \frac{(\lambda z)^x}{x!} ]

Using the exponential series expansion: ( e^\theta = 1 + \theta + \frac{\theta^2}{2!} + \frac{\theta^3}{3!} + \ldots )

We get: ( G(z) = e^{-\lambda} e^{\lambda z} = e^{\lambda(z - 1)} )

⭐ Key Takeaways

The probability generating function is defined for discrete random variables taking nonnegative integer values, expressed as ( G_X(z) = E[z^X] ). For the Bernoulli distribution with parameter p, the PGF simplifies to a linear function of z. For the Poisson distribution with parameter λ, the PGF takes the form ( e^{\lambda(z - 1)} ), derived using the exponential series expansion. The PGF provides a concise way to represent the entire probability mass function and is a fundamental tool for generating moments and studying distributional properties.

🧠 Quick Revision Questions

1. What is the definition of the probability generating function for a discrete random variable X?
2. How do you derive the probability generating function for a Bernoulli random variable with parameter p?
3. Show the step-by-step derivation of the PGF for a Poisson distribution with parameter λ.
4. What exponential series expansion is used to simplify the Poisson PGF?
5. Why is the PGF considered a "power series representation" of the probability mass function?

📘 Lecture 10 — How to utilize Probability Generating Function and Mean and Variance through PGF

📖 Overview: This lecture explains how to recover the probability mass function of a random variable from its probability generating function (PGF) by taking derivatives. It also demonstrates how to calculate the mean and variance of a distribution directly from the PGF, using the Bernoulli and Poisson distributions as key examples.

🗂️ Topics Covered

Recovering the probability mass function from the PGF by evaluating derivatives at zero, illustrated with a Bernoulli random variable. Deriving the mean from the first derivative of the PGF evaluated at 1, with the Poisson distribution as an example. Deriving the variance from the first and second derivatives of the PGF evaluated at 1, also using the Poisson distribution.

📝 Lecture Summary

The probability mass function of X is recovered by taking derivatives of G

The probability mass function of a discrete random variable X can be recovered from its probability generating function (PGF), G(z), using the formula:

🔑 Definition — Recovery of PMF from PGF: ( p(k) = P(X = k) = \frac{G^{(k)}(0)}{k!} )

Where ( G^{(k)}(0) ) is the k-th derivative of G(z) evaluated at z = 0.

📐 Formula: ( P(X = k) = \frac{G^{(k)}(0)}{k!} ) → The probability that X equals k is found by taking the k-th derivative of the PGF, evaluating it at zero, and dividing by k factorial.

Example: Bernoulli Random Variable

The probability generating function of a Bernoulli random variable X with parameter p, where X represents the number of heads obtained when tossing a fair coin once, is given by:

📐 Formula: ( G(z) = \frac{1}{2} + \frac{1}{2}z )

Using the recovery formula, we calculate the probabilities:

For k = 0: [ G^{(0)}(0) = G(0) = \frac{1}{2} + \frac{1}{2}(0) = \frac{1}{2} ] [ P(X = 0) = \frac{G^{(0)}(0)}{0!} = \frac{1/2}{1} = \frac{1}{2} ]

For k = 1: [ G^{(1)}(z) = \frac{1}{2} ] [ G^{(1)}(0) = \frac{1}{2} ] [ P(X = 1) = \frac{G^{(1)}(0)}{1!} = \frac{1/2}{1} = \frac{1}{2} ]

📌 Example: For a fair coin toss, the PGF ( G(z) = \frac{1}{2} + \frac{1}{2}z ) correctly yields ( P(X=0) = \frac{1}{2} ) and ( P(X=1) = \frac{1}{2} ).

MEAN and VARIANCE through the Probability generating function

The mean (expectation) of X can be found directly from the PGF.

🔑 Definition — Mean from PGF: The expectation of X is given by ( E(X) = G'(1^-) ), where ( G'(1^-) = \lim_{z \to 1} G'(z) ) from below.

📐 Formula: ( E(X) = G'(1^-) ) → The mean is the first derivative of the PGF evaluated as z approaches 1 from below.

Example: Poisson Distribution (Mean)

For a Poisson distribution with parameter λ, the PGF is: [ G(z) = e^{\lambda(z - 1)} ]

The first derivative is: [ G'(z) = \lambda e^{\lambda(z - 1)} ]

Evaluating the limit as z approaches 1: [ E(X) = G'(1^-) = \lim_{z \to 1} \lambda e^{\lambda(z - 1)} = \lambda e^{\lambda(1 - 1)} = \lambda e^{0} = \lambda ]

📌 Example: For a Poisson distribution, ( E(X) = \lambda ), confirming that the parameter λ is the mean.

💡 Why this matters: This shows that the PGF method agrees with the known result for the Poisson mean, validating the approach.

The variance of X is also found from the PGF.

🔑 Definition — Variance from PGF: The variance of X is given by ( Var(X) = G''(1^-) + G'(1^-) - [G'(1^-)]^2 )

📐 Formula: ( Var(X) = G''(1^-) + G'(1^-) - [G'(1^-)]^2 ) → The variance requires the second derivative and first derivative of the PGF, both evaluated as z approaches 1 from below.

Example: Poisson Distribution (Variance)

For the Poisson distribution, the second derivative is: [ G''(z) = \lambda^2 e^{\lambda(z - 1)} ]

Evaluating at the limit: [ G''(1^-) = \lambda^2 e^{\lambda(1 - 1)} = \lambda^2 ]

From earlier, ( G'(1^-) = \lambda ).

Applying the formula: [ Var(X) = G''(1^-) + G'(1^-) - [G'(1^-)]^2 ] [ Var(X) = \lambda^2 + \lambda - \lambda^2 ] [ Var(X) = \lambda ]

📌 Example: For a Poisson distribution, ( Var(X) = \lambda ), confirming that the mean and variance are equal in the Poisson distribution.

💡 Why this matters: The PGF provides a unified method for computing both mean and variance, and in this case confirms the well-known property that the Poisson distribution has equal mean and variance.

⭐ Key Takeaways

The probability generating function is a powerful tool that contains all information about a discrete distribution. The PMF is recovered by evaluating derivatives of the PGF at zero and dividing by factorials. The mean is simply the first derivative of the PGF evaluated at 1, while the variance requires both first and second derivatives at 1. The Bernoulli example demonstrates PMF recovery, and the Poisson example confirms that both the mean and variance equal the parameter λ using the PGF method.

🧠 Quick Revision Questions

1. What is the formula to recover ( P(X = k) ) from the probability generating function ( G(z) )?
2. For a Bernoulli random variable with PGF ( G(z) = \frac{1}{2} + \frac{1}{2}z ), what are ( P(X=0) ) and ( P(X=1) )?
3. How is the mean ( E(X) ) expressed in terms of the PGF?
4. What is the formula for the variance ( Var(X) ) in terms of the PGF?
5. For a Poisson distribution, what do both the mean and variance equal when calculated from the PGF?

📘 Lecture 11 — Algebraic Expressions of Some Well-known PGFs

📖 Overview: This lecture presents the Probability Generating Functions (PGFs) for several common probability distributions, including constant, Bernoulli, geometric, binomial, Poisson, negative binomial, and discrete uniform distributions. Understanding these PGFs is essential for deriving moments and characterizing distributions in probability theory.

🗂️ Topics Covered

The lecture begins by listing the PGFs for constant, Bernoulli, geometric, binomial, Poisson, and negative binomial random variables. It then provides a detailed example of deriving the PGF for the discrete uniform distribution using a finite geometric series, specifically for random selection of digits 0 through 9.

📝 Lecture Summary

Algebraic Expressions of Some Well-known PGFs

For some of the more common distributions, the PGFs are as follows:

(i) Constant r.v. — If (P(X=c) = 1) where (c \neq 0), then the PGF is: [ G_X(z) = E[z^X] = z^c ]

🔑 Definition — Constant Random Variable PGF: For a constant random variable that always takes the value (c), the PGF is simply (z^c).

(ii) Bernoulli r.v. — If (P(X=1) = p) and (P(X=0) = q = 1-p), where (p) is the probability of success and (q) is the probability of failure, then: [ G_X(z) = E[z^X] = q + pz ]

🔑 Definition — Bernoulli PGF: The PGF for a Bernoulli random variable is a linear function (q + pz), representing the weighted average of (z^0) and (z^1).

(iii) Geometric r.v. — If (P(X=k) = pq^{k-1}) for (k=1,2,\ldots) where (q=1-p), then: [ G_X(z) = \frac{pz}{1 - qz} \quad \text{if } |z| < \frac{1}{q} ]

🔑 Definition — Geometric PGF: The PGF for a geometric distribution is a rational function (\frac{pz}{1-qz}), valid for (|z| < 1/q).

(iv) Binomial r.v. — If (X \sim \text{Bin}(n,p)), then: [ G_X(z) = (q + pz)^n ] where (q = 1-p).

🔑 Definition — Binomial PGF: The PGF for a binomial distribution is the (n)-th power of the Bernoulli PGF, ((q+pz)^n).

(v) Poisson r.v. — If (X \sim \text{Poisson}(\lambda)), then: [ G_X(z) = e^{\lambda(z-1)} = \sum_{k=0}^\infty \frac{e^{-\lambda} \lambda^k}{k!} z^k ]

🔑 Definition — Poisson PGF: The PGF for a Poisson distribution is the exponential function (e^{\lambda(z-1)}), which arises from its probability mass function.

(vi) Negative Binomial r.v. — If (X \sim \text{NB}(n,p)), then: [ G_X(z) = \left( \frac{p}{1 - qz} \right)^n \quad \text{if } |z| < \frac{1}{q} ] where (q = 1-p) and (n) is the number of successes.

🔑 Definition — Negative Binomial PGF: The PGF for a negative binomial distribution is the (n)-th power of the geometric PGF's numerator/denominator form, (\left(\frac{p}{1-qz}\right)^n).

💡 Why this matters: Each of these PGFs encodes the entire probability distribution in a compact algebraic form, making it easy to compute moments and other properties.

Discrete Uniform Distribution

Example: Random selection of one of the ten digits 0,1,2,...,9.

For this distribution, the probability mass function is uniform:

• (P(X=x) = 1/10) for (x=0,1,2,\ldots,9)

The PGF is given by: [ G_X(z) = E[z^X] = \sum_{x=0}^9 z^x \cdot \frac{1}{10} = \frac{1}{10} (1 + z + z^2 + z^3 + \ldots + z^9) ]

This is a finite geometric series with:

• First term (a = 1)
• Common ratio (r = z)
• Number of terms (n = 10)

Formula for sum of finite geometric series: When (r \neq 1), [ \text{Sum} = a \cdot \frac{1 - r^n}{1 - r} ]

📐 Formula: For a geometric series with first term (a), common ratio (r), and (n) terms: [ S_n = a \cdot \frac{1 - r^n}{1 - r} \quad \text{(for } r \neq 1\text{)} ] → This gives the sum of the first (n) terms.

Applying this formula: [ 1 + z + z^2 + \ldots + z^9 = \frac{1 - z^{10}}{1 - z} ]

Therefore: [ G_X(z) = \frac{1}{10} \cdot \frac{1 - z^{10}}{1 - z} ]

📌 Example: For the discrete uniform distribution over ({0,1,\ldots,9}):

• Each outcome has probability (1/10)
• The PGF is (G_X(z) = \frac{1-z^{10}}{10(1-z)})
• For example, if (z=2), then (G_X(2) = \frac{1-2^{10}}{10(1-2)} = \frac{1-1024}{10(-1)} = \frac{-1023}{-10} = 102.3), which represents (E[2^X])

💡 Why this matters: The discrete uniform distribution PGF shows how finite geometric series naturally arise when dealing with uniform probabilities over a finite set of consecutive integers.

⭐ Key Takeaways

The PGFs for constant, Bernoulli, geometric, binomial, Poisson, and negative binomial distributions have been explicitly derived, each taking a specific algebraic form (polynomial, rational, exponential, or power). The discrete uniform distribution yields a PGF expressed as a finite geometric series, leading to the closed form (\frac{1}{10}\cdot\frac{1-z^{10}}{1-z}) for the digits 0-9. These PGFs serve as powerful tools for moment generation and distribution characterization. Recognizing the pattern of PGFs (e.g., binomial as a power of Bernoulli) aids in understanding relationships between distributions. The finite geometric series formula (\frac{1-r^n}{1-r}) is essential for deriving PGFs of uniform distributions over consecutive integers.

🧠 Quick Revision Questions

1. What is the PGF of a Bernoulli random variable with success probability (p)?
2. Derive the PGF of a geometric distribution with parameter (p).
3. How is the PGF of a binomial distribution related to the Bernoulli PGF?
4. Write the PGF of a Poisson distribution with mean (\lambda).
5. For a discrete uniform distribution over ({0,1,2,\ldots,9}), what is the closed-form expression of its PGF?

📘 Lecture 12 — A Linear Combination of PDFs is a PDF

📖 Overview: This lecture establishes that a linear combination of probability density functions (PDFs), where the coefficients are nonnegative and sum to 1, is itself a valid probability density function. It also proves that the mean of such a linear combination equals the same linear combination of the individual means. This concept is fundamental for constructing mixture distributions.

🗂️ Topics Covered

The lecture introduces the concept that a linear combination of continuous-type PDFs (with nonnegative coefficients summing to 1) produces a new PDF. It then proves this by showing the integral of the weighted sum equals 1. Next, the lecture proves that the expected value (mean) of this new distribution is a linear combination of the individual means. The key result is that both the PDF and its mean preserve the linear combination property.

📝 Lecture Summary

Consider k continuous-type distributions

Let there be k continuous-type distributions, each with its own probability density function ( f_i(x) ) and mean ( \mu_i ), for ( i = 1, 2, \ldots, k ). Let ( c_i \ge 0 ) be constants for ( i = 1, 2, \ldots, k ), such that ( c_1 + c_2 + \ldots + c_k = 0 + \ldots + 0 = 1 ). Define a new function ( w(x) = c_1 f_1(x) + c_2 f_2(x) + \ldots + c_k f_k(x) ). The lecture aims to show that ( w(x) ) is a valid PDF.

🔑 Definition — Linear combination: An expression constructed from a set of terms by multiplying each term by a constant and adding those results. For example, ( ax + by ) is a linear combination of ( x ) and ( y ).

Proof: A Linear Combination of PDFs is a PDF

To prove that ( w(x) = c_1 f_1(x) + c_2 f_2(x) + \ldots + c_k f_k(x) ) is a valid PDF, we must show its integral over all real numbers equals 1. The left-hand side of the integral is: [ \int_{-\infty}^{\infty} w(x) dx = \int_{-\infty}^{\infty} [c_1 f_1(x) + c_2 f_2(x) + \ldots + c_k f_k(x)] dx ] Since integration is a linear operator, this becomes: [ = c_1 \int_{-\infty}^{\infty} f_1(x) dx + c_2 \int_{-\infty}^{\infty} f_2(x) dx + \ldots + c_k \int_{-\infty}^{\infty} f_k(x) dx ] Since each ( f_i(x) ) is a PDF, the integral of each is 1: ( \int_{-\infty}^{\infty} f_i(x) dx = 1 ). Therefore: [ = c_1 (1) + c_2 (1) + \ldots + c_k (1) ] [ = c_1 + c_2 + \ldots + c_k ] By our choice of constants, ( c_1 + c_2 + \ldots + c_k = 1 ). Hence, the left-hand side equals the right-hand side: [ \int_{-\infty}^{\infty} w(x) dx = 1 ] This proves ( w(x) ) is a valid PDF. In other words, the mean of a linear combination of PDFs is that same linear combination of the means of those PDFs.

📐 Formula: ( w(x) = c_1 f_1(x) + c_2 f_2(x) + \ldots + c_k f_k(x) ) → A weighted sum of PDFs, where weights are nonnegative and sum to 1, creating a new PDF.

Show that the mean of the distribution having pdf ( w(x) ) is ( \sum_{i=1}^{k} c_i \mu_i )

The mean (expected value) of the new distribution with PDF ( w(x) ) is: [ E[X] = \int_{-\infty}^{\infty} x w(x) dx = \int_{-\infty}^{\infty} x [c_1 f_1(x) + c_2 f_2(x) + \ldots + c_k f_k(x)] dx ] Again, using linearity of integration: [ = c_1 \int_{-\infty}^{\infty} x f_1(x) dx + c_2 \int_{-\infty}^{\infty} x f_2(x) dx + \ldots + c_k \int_{-\infty}^{\infty} x f_k(x) dx ] The expected value of ( X ) for each individual distribution is ( \int_{-\infty}^{\infty} x f_i(x) dx = \mu_i ). Therefore: [ = c_1 \mu_1 + c_2 \mu_2 + \ldots + c_k \mu_k ] Hence, we get: [ E[X] = \sum_{i=1}^{k} c_i \mu_i ]

💡 Why this matters: This property allows us to model complex distributions by mixing simpler ones. For example, if we have data that appears to come from two different populations (like heights of men and women), we can represent the overall distribution as a weighted average of the two individual distributions.

📐 Formula: ( E[X] = \sum_{i=1}^{k} c_i \mu_i ) → The mean of a linear combination of PDFs is the same linear combination of their individual means.

📌 Example: Given three PDFs with means ( \mu_1 = 5 ), ( \mu_2 = 10 ), ( \mu_3 = 15 ) and coefficients ( c_1 = 0.5 ), ( c_2 = 0.3 ), ( c_3 = 0.2 ) (note: ( 0.5 + 0.3 + 0.2 = 1 )), the mean of the mixture distribution ( w(x) ) is: [ E[X] = 0.5(5) + 0.3(10) + 0.2(15) = 2.5 + 3.0 + 3.0 = 8.5 ]

⭐ Key Takeaways

The most critical concept is that a linear combination of PDFs, with nonnegative weights summing to 1, is itself a valid PDF. This is proven by showing its total integral equals 1. Additionally, the mean (expected value) of this weighted PDF is the same weighted sum of the individual means. The conditions for the constants ( c_i ) being nonnegative and summing to 1 are essential for the resulting function to be a legitimate probability distribution. This principle allows us to construct and analyze mixture distributions by combining simpler component distributions.

🧠 Quick Revision Questions

1. What are the two conditions the constants ( c_i ) must satisfy for ( w(x) = \sum c_i f_i(x) ) to be a PDF?
2. Explain why the integral of ( w(x) ) equals 1.
3. How is the mean of a mixture distribution related to the means of its component distributions?
4. If ( w(x) = 0.7 f_1(x) + 0.3 f_2(x) ) and ( \mu_1 = 20, \mu_2 = 50 ), what is the mean of ( w(x) )?
5. Why is the property of linearity of integrals crucial in proving both results in this lecture?

📘 Lecture 13 — Concept of Cumulative Distribution Function (discrete and continuous)

📖 Overview: This lecture introduces the Cumulative Distribution Function (CDF), a fundamental concept in probability theory that describes the probability that a random variable takes a value less than or equal to a specific point. The lecture covers the definition, properties, and graphical representation of CDFs for both discrete and continuous random variables, with practical examples to illustrate the differences.

🗂️ Topics Covered

The lecture begins with the formal definition of the Cumulative Distribution Function (CDF) for any random variable. It then explains the graphical representation of CDFs, noting that for discrete random variables the CDF forms a step function (staircase), while for continuous random variables it forms a continuous curve or line. Two detailed examples are presented: one for a discrete random variable with a probability mass function (pmf) and another for a continuous random variable uniformly distributed between 0 and 2, showing how to derive and sketch the CDF in each case.

📝 Lecture Summary

Definition: (Cumulative Distribution Function)

Let X be a random variable. Its cumulative distribution function (cdf) is defined by Fₓ(x), given by:

Fₓ(x) = P(X ≤ x) for -∞ < x < ∞

As above, we shorten: Fₓ(x) = P(X ≤ x)

Also, Fₓ(x) is often called simply the distribution function (df).

The graph for the discrete case is like a staircase, which is also called a Step function. On the other hand, in the continuous case it is a continuous curve or a line.

🔑 Definition — Cumulative Distribution Function (CDF): The function Fₓ(x) = P(X ≤ x) that gives the probability that the random variable X takes a value less than or equal to x.

Example 1: Discrete Random Variable

Let pₓ(x) be the pmf of a random variable X. Find the cdf F(x) of X and sketch its graph with that of pₓ(x) if: a) pₓ(x) = 1/3, x = -1, 0, 1, zero elsewhere.

The probability mass function assigns equal probability of 1/3 to each of the three values: -1, 0, and 1.

📌 Example: For X with pmf pₓ(-1) = 1/3, pₓ(0) = 1/3, pₓ(1) = 1/3:

• For x < -1: F(x) = 0
• For -1 ≤ x < 0: F(x) = P(X ≤ -1) = 1/3
• For 0 ≤ x < 1: F(x) = P(X ≤ 0) = P(X = -1) + P(X = 0) = 1/3 + 1/3 = 2/3
• For x ≥ 1: F(x) = P(X ≤ 1) = 1/3 + 1/3 + 1/3 = 1

The graph is a step function with jumps at x = -1, 0, and 1.

Figure 1: Distribution Function - shows the staircase pattern for discrete case.

Example 2: Continuous Random Variable

Let X denote a real number chosen at random between 0 and 2. We now obtain the cdf of X.

First, if x < 0, then P(X ≤ x) = 0. Next, if x ≥ 2, then P(X ≤ x) = 1. Finally, if 0 < x < 2, it follows from equation (2) that P(X ≤ x) = P(0 < X ≤ x) = x - 0 = x.

Hence the cdf of X is:

Fₓ(x) =

• 0, if x < 0
• x/2, if 0 ≤ x < 2
• 1, if x ≥ 2

Note: For a uniform distribution on interval [a, b], P(a < X < b) = (b - a) / (b - a) = 1, and for any subinterval [c, d] within [a, b], P(c < X < d) = (d - c) / (b - a).

📐 Formula: For a uniform distribution on [a, b]: P(c < X < d) = (d - c) / (b - a) for a < c < d < b.

📌 Example: For X uniformly distributed on [0, 2]:

• Fₓ(0.5) = P(X ≤ 0.5) = 0.5/2 = 0.25
• Fₓ(1) = 1/2 = 0.5
• Fₓ(1.5) = 1.5/2 = 0.75

Figure 2: Distribution Function - shows the continuous linear graph from (0,0) to (2,1).

💡 Why this matters: The CDF is a complete description of a random variable's probability distribution. Unlike the pmf or pdf, the CDF is defined for all real numbers and always exists for any random variable, making it a universal tool for probability calculations.

⭐ Key Takeaways

The Cumulative Distribution Function Fₓ(x) = P(X ≤ x) is defined for all real numbers and always ranges from 0 to 1 as x increases from negative infinity to positive infinity. For discrete random variables, the CDF is a step function that jumps at each point where the pmf is nonzero, with the jump height equal to the probability at that point. For continuous random variables, the CDF is a continuous function that can be obtained by integrating the probability density function. The CDF is non-decreasing, meaning if x₁ < x₂, then F(x₁) ≤ F(x₂). The CDF is fundamental because it provides a unified way to compute probabilities: P(a < X ≤ b) = F(b) - F(a) for any a < b.

🧠 Quick Revision Questions

1. What is the formal definition of the Cumulative Distribution Function (CDF)?
2. How does the graph of a CDF differ between a discrete random variable and a continuous random variable?
3. For the discrete example with pₓ(x) = 1/3 at x = -1, 0, 1, what is F(0.5)?
4. For a continuous uniform random variable on [0, 2], what is the CDF value at x = 1.5?
5. What property must a CDF always satisfy regarding monotonicity?

📘 Lecture 14 — Example of the CDF of a Discrete Random Variable

📖 Overview: This lecture demonstrates how to derive and verify the cumulative distribution function (CDF) for a discrete random variable using a concrete example involving drawing slips of paper numbered 1 to 100 from an urn. It shows the step-by-step process of defining the probability mass function and then deriving the CDF formula, validating its correctness.

🗂️ Topics Covered

The lecture covers a single worked example of finding the CDF for a discrete random variable X representing the number on a slip drawn from an urn containing slips numbered 1 through 100, with i slips bearing the number i. It demonstrates the probability mass function (PMF) derivation and the formula for the cumulative distribution function F(x) = x/10100 for 1 ≤ x ≤ 100.

📝 Lecture Summary

Example — Setting Up the Problem

Consider an urn containing slips of paper, each with one of the numbers 1, 2, ..., 100. There are i slips with the number i on them for i = 1, 2, ..., 100. For example, there are 25 slips with the number 25. The slips are identical except for the numbers. One slip is drawn at random. Let X be the number on the slip.

The total number of slips in the urn is the sum of integers from 1 to 100: 1 + 2 + 3 + ... + 100 = 5050 slips.

The probability that the drawn slip shows the number x is the number of slips with that number divided by the total number of slips.

🔑 Definition — Probability Mass Function (PMF): For a discrete random variable X, p(x) = P(X = x) = (number of favorable outcomes) / (total number of outcomes)

📌 Example for PMF: The PMF for this problem is:

• p(1) = 1/5050, p(2) = 2/5050, p(3) = 3/5050, ..., p(100) = 100/5050
• Sum of all probabilities = 5050/5050 = 1 ✓ (confirms it is a valid PMF)

Deriving the CDF

The cumulative distribution function (CDF) F(x) = P(X ≤ x) for a discrete random variable is the sum of probabilities for all values less than or equal to x.

For this problem, F(x) = P(X ≤ x) = (1 + 2 + 3 + ... + [x]) / 5050, where [x] is the greatest integer less than or equal to x (the floor function).

🔑 Definition — Greatest integer in x ([x]): The largest integer that is less than or equal to x. For example, [25.7] = 25, [50] = 50, [8.1] = 8.

The sum of the first n positive integers is given by the formula: 1 + 2 + 3 + ... + n = n(n+1)/2.

Substituting n = [x]: 1 + 2 + 3 + ... + [x] = x/2

Therefore: F(x) = x / (2 × 5050) = x / 10100

📐 Formula: F(x) = x/10100, for 1 ≤ x ≤ 100, where [x] is the greatest integer in x.

💡 Why this matters: This formula allows us to calculate the probability that X takes a value less than or equal to any given x without summing individual probabilities each time.

📌 Example: To find P(X ≤ 25): F(25) = 25/10100 = 25 × 26 / 10100 = 650/10100 = 65/1010 = 13/202 ≈ 0.06436

📌 Example: To find P(X ≤ 50): F(50) = 50/10100 = 50 × 51 / 10100 = 2550/10100 = 255/1010 = 51/202 ≈ 0.2525

📌 Verification: At x = 100: F(100) = 100/10100 = 100 × 101 / 10100 = 10100/10100 = 1 ✓

This confirms the CDF is valid since the total probability equals 1 for the maximum value of X.

⭐ Key Takeaways

The lecture demonstrates how to derive a CDF for a discrete random variable when the PMF follows a pattern. The total number of outcomes must first be computed (here, 5050 slips). The PMF is p(i) = i/5050 for i = 1 to 100. The CDF at any x is the sum of probabilities for all values ≤ x, which reduces to the sum of the first [x] integers divided by 5050, yielding the closed form F(x) = x/10100. Always verify the CDF equals 1 at the maximum value.

🧠 Quick Revision Questions

1. What is the total number of slips in the urn in this example?
2. What is the probability that the drawn slip shows the number 30?
3. What does the notation [x] mean in the CDF formula?
4. What is F(75) using the derived formula? Show your work.
5. How can you verify that F(x) is a valid cumulative distribution function for this discrete random variable?

📘 Lecture 15 — Obtaining the PMF from the CDF

📖 Overview: This lecture teaches how to derive the probability mass function (PMF) from a given cumulative distribution function (CDF). This is a fundamental skill in probability theory, as it allows one to reverse-engineer the underlying discrete probability distribution.

🗂️ Topics Covered

The lecture focuses on a single worked example: given a CDF defined as ( F(x) = \sum_{j=1}^x \left(\frac{1}{2}\right)^j ), find its corresponding PMF. It systematically demonstrates how to compute the probability at each individual point by finding the difference between successive CDF values.

📝 Lecture Summary

Obtaining the PMF from the CDF

The lecture presents a method for obtaining the PMF from a CDF for discrete random variables. The key principle is that for a discrete random variable, the PMF at a specific point (x) is the difference between the CDF at (x) and the CDF at the point immediately before (x). This is because the CDF accumulates all probability up to and including (x), while the PMF gives the probability exactly at (x).

Given the CDF ( F(x) = \sum_{j=1}^x \left(\frac{1}{2}\right)^j ), the PMF ( f(x) ) is found using the relation ( f(x) = F(x) - F(x-1) ). The lecture then computes the PMF for various values of (x):

• For (x=1): ( f(1) = F(1) - F(0) = \frac{1}{2} - 0 = \frac{1}{2} )
• For (x=2): ( f(2) = F(2) - F(1) = \left(\frac{1}{2} + \frac{1}{4}\right) - \frac{1}{2} = \frac{1}{4} )
• For (x=3): ( f(3) = F(3) - F(2) = \left(\frac{1}{2} + \frac{1}{4} + \frac{1}{8}\right) - \left(\frac{1}{2} + \frac{1}{4}\right) = \frac{1}{8} )

The pattern continues such that the PMF is ( f(x) = \left(\frac{1}{2}\right)^x ). 💡 Why this matters: This pattern reveals that each successive probability is half the previous one, confirming that the CDF defines a geometric distribution with parameter (p = \frac{1}{2}).

💡 Why this matters: The relationship ( f(x) = F(x) - F(x-1) ) is a cornerstone for working with discrete distributions. It allows you to reconstruct the entire probability distribution from the cumulative function.

🔑 Definition — PMF: The probability mass function of a discrete random variable gives the probability that the variable is exactly equal to a specific value. 📐 Formula: For a discrete random variable (X), the PMF (f(x)) from the CDF (F(x)) is ( f(x) = F(x) - F(x-1) ) → The probability at exactly (x) is the cumulative probability up to (x) minus the cumulative probability up to just before (x). 📌 Example: Given ( F(x) = \sum_{j=1}^x \left(\frac{1}{2}\right)^j ), find ( f(2) ):

• Compute ( F(2) = \frac{1}{2} + \frac{1}{4} = \frac{3}{4} )
• Compute ( F(1) = \frac{1}{2} )
• Then ( f(2) = F(2) - F(1) = \frac{3}{4} - \frac{1}{2} = \frac{1}{4} )

⭐ Key Takeaways

The most critical concept from this lecture is that for discrete random variables, the PMF can be directly obtained from the CDF by taking successive differences: ( f(x) = F(x) - F(x-1) ). This relationship is the inverse of the CDF accumulation process. The example demonstrated that the CDF ( F(x) = \sum_{j=1}^x (1/2)^j ) yields the PMF ( f(x) = (1/2)^x ), which is a geometric distribution. Understanding this one-to-one correspondence between CDF and PMF is essential for analyzing discrete probability distributions.

🧠 Quick Revision Questions

1. What is the general formula for obtaining the PMF from a CDF for a discrete random variable?
2. Given the CDF ( F(x) = \sum_{j=1}^x \left(\frac{1}{2}\right)^j ), what is the value of ( f(4) )?
3. Why is it necessary to consider ( F(x-1) ) when finding the PMF at point (x)?
4. In the example from this lecture, what probability distribution does the PMF ( f(x) = (1/2)^x ) represent?
5. If the CDF of a discrete random variable is ( F(x) = 1 - (1/3)^x ) for (x = 1, 2, 3, ...), what would be the corresponding PMF?

📘 Lecture 16 — Example of the CDF of a Continuous Random Variable

📖 Overview: This lecture provides an illustrative example of the Cumulative Distribution Function (CDF) for a continuous random variable. It then introduces the Exponential Distribution, presenting both its Probability Density Function (PDF) and CDF for different mean values (0.5, 1, and 1.5), showing how the distribution changes with the parameter.

🗂️ Topics Covered

This lecture covers an example of the CDF of a continuous random variable where the function is piecewise defined over intervals. It then covers the PDF of the Exponential Distribution for three different mean values of 0.5, 1, and 1.5, followed by the CDF of the Exponential Distribution for the same mean values.

📝 Lecture Summary

Example of the CDF of a Continuous Random Variable

For a continuous random variable X, the Cumulative Distribution Function (CDF) is given as a piecewise function. Specifically, F(x) = 0 for all x values which are less than zero. F(x) = x for all those x values that lie between 0 and 1. F(x) = 1 for all those x values that are greater than 1.

🔑 Definition — Cumulative Distribution Function (CDF): A function F(x) that gives the probability that a random variable X is less than or equal to a specific value x. 📐 Formula: F(x) = P(X ≤ x) → The probability that X is less than or equal to x. 📌 Example: For the given continuous random variable X, to find P(X ≤ 0.5): Since 0.5 lies between 0 and 1, F(0.5) = 0.5. Therefore, the probability that X is less than or equal to 0.5 is 0.5.

PDF of the Exponential Distribution with mean=0.5, 1, 1.5

The Probability Density Function (PDF) of the Exponential Distribution is presented for three different mean values. The mean is represented by the parameter. The general form of the PDF is f(x) = (1/mean) * e^(-x/mean) for 0 < x < ∞. For mean = 0.5, the PDF is f(x) = (1/0.5) * e^(-x/0.5), displayed in orange. For mean = 1, the PDF is f(x) = (1/1) * e^(-x/1) = e^(-x), displayed in purple. For mean = 1.5, the PDF is f(x) = (1/1.5) * e^(-x/1.5), displayed in sky blue. The graph shows how the shape of the exponential distribution varies with different mean values. 💡 Why this matters: The mean parameter directly affects the rate of decay of the exponential distribution.

🔑 Definition — Probability Density Function (PDF): A function f(x) that describes the relative likelihood of a continuous random variable taking on a given value. 📐 Formula: For Exponential Distribution: f(x) = (1/μ) e^(-x/μ) for x ≥ 0 → The probability density at point x depends on the mean μ. 📌 Example: For the Exponential Distribution with mean=1, the PDF is f(x) = e^(-x) for 0 < x < ∞. This specific distribution is shown in purple.

CDF of the Exponential Distribution with mean=0.5, 1, 1.5

The Cumulative Distribution Function (CDF) of the Exponential Distribution is presented for the same three mean values. The general formula for the CDF of an exponential distribution is F(x) = 1 - e^(-x/mean) for 0 < x < ∞. For mean = 0.5, the CDF is F(x) = 1 - e^(-x/0.5). For mean = 1, the CDF is F(x) = 1 - e^(-x/1) = 1 - e^(-x). For mean = 1.5, the CDF is F(x) = 1 - e^(-x/1.5). The CDF provides the cumulative probability up to a given value x.

🔑 Definition — CDF of Exponential Distribution: F(x) = 1 - e^(-x/μ) for x ≥ 0 → The probability that X is less than or equal to x. 📐 Formula: F(x) = 1 - e^(-x/μ) → Gives P(X ≤ x) for an exponential random variable with mean μ. 📌 Example: For the Exponential Distribution with mean=1, the CDF is F(x) = 1 - e^(-x). To find P(X ≤ 1), we compute F(1) = 1 - e^(-1) = 1 - 0.3679 = 0.6321. Therefore, the probability that X is less than or equal to 1 is approximately 0.6321.

⭐ Key Takeaways

The CDF for a continuous random variable can be a piecewise function, ranging from 0 to 1 and increasing as x increases. The Exponential Distribution is defined by its mean parameter, and its PDF has the form (1/mean)*e^(-x/mean). The corresponding CDF of the Exponential Distribution is given by F(x) = 1 - e^(-x/mean), which is used to calculate cumulative probabilities. Different mean values (0.5, 1, 1.5) produce different shapes for both the PDF and CDF, demonstrating how the parameter influences the spread of the distribution. Understanding the relationship between the PDF and CDF is essential for computing probabilities for exponential random variables.

🧠 Quick Revision Questions

1. What is the value of the CDF, F(x), for a continuous random variable when x is less than 0, according to the example given?
2. Write the general formula for the probability density function (PDF) of the Exponential Distribution.
3. What is the CDF of the Exponential Distribution with mean = 1.5?
4. If an exponential random variable has a mean of 0.5, what is its PDF function?
5. Using the CDF formula, calculate P(X ≤ 0.5) for an Exponential Distribution with mean = 1.

📘 Lecture 17 — Concept of two Random Variables being Equal in Distribution

📖 Overview: This lecture introduces the concept of two random variables being equal in distribution, which is a weaker form of equality than actual identity. It clarifies that two random variables can have the same distribution function even when they are different random variables. An example using the uniform distribution illustrates this important distinction.

🗂️ Topics Covered

Definition of equality in distribution for two random variables; notation for equality in distribution; distinction between equality in distribution and actual equality of random variables; example illustrating that X and Y = 1-X are different but equal in distribution; calculation of CDF for transformed variables; conceptual clarification that equal distribution does not imply identical random variables.

📝 Lecture Summary

Equality in Distribution — Definition and Concept

We first consider the situation when two random variables X and Y are not equal in distribution: Fₓ(x) ≠ Fᵧ(x). We say that X and Y are equal in distribution if and only if Fₓ(x) = Fᵧ(x), for all x ∈ ℝ. Mathematically we express it as X ≜ Y (or X = D Y).

💡 Why this matters: It is important to note that while X and Y may be equal in distribution, they may be quite different as actual random variables.

🔑 Definition — Equality in Distribution: Two random variables X and Y are equal in distribution if and only if their cumulative distribution functions are identical for all real numbers x: Fₓ(x) = Fᵧ(x), ∀ x ∈ ℝ.

📌 Example: Recall that X denotes a real number chosen at random between 0 and 1. We obtain the CDF of X. First, if x < 0, then P(X ≤ x) = 0. Next, if x ≥ 1, then P(X ≤ x) = 1. Finally, for 0 < x < 1, it follows that P(X ≤ x) = x - 0 = x. Hence the CDF of X is:

Fₓ(x) =

• 0, if x < 0
• x, if 0 ≤ x < 1
• 1, if x ≥ 1

Example with Transformation Y = 1 - X

For instance, in the above example define the random variable Y and transform it as Y = 1 - X. Then Y ≠ X. For example, let X = 0.2 then Y = 1 - 0.2 = 0.8. So, when X is 0.2 then Y is 0.8 which is not the same. Therefore it is interesting to note that the space of Y is the interval (0,1), the same as X.

Computing probabilities: F(0.8) = 1 - P(X < 1 - 0.8) = 1 - P(X < 0.2) = 1 - 0.2 = 0.8. F(0.6) = 1 - P(X < 1 - 0.6) = 1 - P(X < 0.4) = 1 - 0.4 = 0.6.

But we know that for the Uniform distribution defined on (0,1), P(X < 0.2) = 0.2. Therefore, P(X < 1 - y) = 1 - y.

Further, the CDF of Y is 0 for y < 0; 1 for y ≥ 1; and for 0 ≤ y < 1, it is:

Fᵧ(y) = P(Y ≤ y) = P(1 - X ≤ y) = P(X ≥ 1 - y) = 1 - P(X < 1 - y) = 1 - (1 - y) = y

Hence, Fᵧ(y) = y, for 0 < y < 1, which is of exactly the same form as the CDF of X.

📐 Formula: For Y = 1 - X, where X ~ Uniform(0,1), the CDF of Y is Fᵧ(y) = y for 0 ≤ y < 1, which is identical to Fₓ(x) = x for 0 ≤ x < 1.

Therefore, it is clarified that two random variables may not be the same yet, sometimes they may be equal in distribution: i.e., Y ≜ X, but Y ≠ X.

⭐ Key Takeaways

The most critical concept from this lecture is that equality in distribution means two random variables have identical cumulative distribution functions, but this does NOT mean they are the same random variable. Using the uniform(0,1) example, X and Y = 1-X are clearly different — when X = 0.2, Y = 0.8 — yet their CDFs are identical. This shows that distributional equality is a property of probability laws, not of the random variables themselves. Students must remember the notation X ≜ Y and understand that the CDF completely characterizes distribution. The key distinction between "equal as random variables" and "equal in distribution" is fundamental for advanced probability and statistics.

🧠 Quick Revision Questions

1. What is the formal mathematical definition of two random variables being equal in distribution?
2. Does X ≜ Y imply that X = Y? Explain with an example.
3. For X ~ Uniform(0,1) and Y = 1 - X, what is Fᵧ(0.3) and Fᵧ(0.7)?
4. Why must the equality Fₓ(x) = Fᵧ(x) hold for ALL real x, not just some values?
5. If two random variables have the same probability density function, are they necessarily equal in distribution? Are they necessarily the same random variable?

📘 Lecture 18 — First property of Cumulative Distribution Function and its Proof

📖 Overview: This lecture introduces the first fundamental property of the Cumulative Distribution Function (CDF) — the property of nondecreasing. Through a formal theorem and proof, it demonstrates that if a random variable has a CDF F(x), then for any two values a and b where a < b, the corresponding probabilities must satisfy F(a) ≤ F(b). This property is essential for understanding how cumulative probabilities behave as we move along the real number line.

🗂️ Topics Covered

The lecture covers Theorem 1 concerning the first property of the Cumulative Distribution Function, specifically the nondecreasing property. It presents the formal statement of the theorem along with its complete proof, which relies on the subset relationship between events and the probability monotonicity theorem.

📝 Lecture Summary

Theorem 1: First Property of Cumulative Distribution Function

Let X be a random variable with cumulative distribution function F(x). Then part (a) states: For all a and b, if a < b, then F(a) ≤ F(b). This means F is nondecreasing.

The proof of part (a) relies on the subset relationship between events. Because a < b, we can observe that the interval {X ≤ a} is a subset of {X ≤ b}. In other words, {X ≤ a} ⊆ {X ≤ b}.

The theorem states: If C₁ and C₂ are events such that C₁ ⊆ C₂, then P(C₁) ≤ P(C₂).

💡 Why this matters: This probability monotonicity theorem is the fundamental logical link that transforms the set inclusion relationship into an inequality of probabilities.

Applying this theorem, since the interval X ≤ a is contained in the interval X ≤ b, therefore the probability of X ≤ a is less than or equal to the probability of X ≤ b. This gives us F(a) ≤ F(b), which directly proves the nondecreasing property.

Example: Consider a random variable X with CDF F(x). Let a = 2 and b = 5, so a < b. The event {X ≤ 2} is contained within {X ≤ 5} because any value less than or equal to 2 is automatically less than or equal to 5. Therefore, by the theorem, P(X ≤ 2) ≤ P(X ≤ 5), which means F(2) ≤ F(5).

⭐ Key Takeaways

The first property of the Cumulative Distribution Function establishes that F(x) is nondecreasing — as the value of x increases, the cumulative probability cannot decrease. This property is proved using the fundamental theorem that if one event is a subset of another event, the probability of the subset event is less than or equal to the probability of the superset event. The proof depends on recognizing that for a < b, the event {X ≤ a} is always contained in {X ≤ b} because any x satisfying the first inequality automatically satisfies the second. This property is essential for validating that any proposed function can serve as a legitimate CDF.

🧠 Quick Revision Questions

1. State the first property of the Cumulative Distribution Function as given in Theorem 1.
2. What is the set inclusion relationship between {X ≤ a} and {X ≤ b} when a < b?
3. What theorem connects set inclusion to probability inequality in this proof?
4. If F(3) = 0.4 and F(7) = 0.4, does this violate the nondecreasing property? Explain.
5. Why must the interval X ≤ a be a subset of X ≤ b when a < b?

📘 Lecture 19 — Second property of Cumulative Distribution Function and its Proof

📖 Overview: This lecture focuses on the second property of the cumulative distribution function (CDF), which states that the lower limit of F(x) as x approaches negative infinity is 0. The lecture provides a conceptual explanation and a formal mathematical proof of this essential property.

🗂️ Topics Covered

The lecture covers Theorem 1(b) regarding the CDF, explaining that as x approaches negative infinity, F(x) approaches 0. It provides both a conceptual understanding based on the definition of F(x) as P(X < x) and an impossible event, and a mathematical formulation using limits.

📝 Lecture Summary

Theorem 1(b): The Lower Limit of F(x) is 0

Let X be a random variable with cumulative distribution function F(x). Then, property (b) states: lim_{x→-∞} F(x) = 0 (the lower limit of F is 0).

We know that F(x) = P(X < x), so obviously F(-∞) = P(X < -∞), which is an impossible event. Therefore, F(-∞) = 0.

Mathematically, we write: lim_{x→-∞} F(x) = 0.

💡 Why this matters: This property establishes that as we move to the extreme left of the distribution (towards negative infinity), the cumulative probability becomes zero — meaning no probability mass exists below any finite lower bound in a proper probability distribution.

⭐ Key Takeaways

The cumulative distribution function F(x) has the property that its lower limit is zero as x approaches negative infinity. This is because F(x) = P(X < x), and the probability of X being less than negative infinity is an impossible event, which has probability zero. Mathematically, this is expressed as lim_{x→-∞} F(x) = 0. This property is fundamental for understanding the behavior of probability distributions at their lower extremes and ensures that all probability is accumulated as x increases.

🧠 Quick Revision Questions

1. What does the notation F(x) represent in probability theory?
2. What is the second property of the cumulative distribution function discussed in this lecture?
3. Why does F(-∞) equal 0? Explain in terms of probability theory.
4. How is the second property of the CDF expressed mathematically using limits?
5. What type of event is represented by P(X < -∞), and what is its probability?

📘 Lecture 20 — Third property of Cumulative Distribution Function and its Proof

📖 Overview: This lecture presents the third property of the cumulative distribution function (CDF), demonstrating that as x approaches infinity, the CDF approaches 1. This property establishes that the upper limit of any cumulative distribution function is always 1, confirming that the total probability under any probability distribution equals 1.

🗂️ Topics Covered

The lecture covers the third property of the cumulative distribution function F(x), specifically proving that the limit of F(x) as x approaches infinity equals 1. It provides both a mathematical proof using probability concepts and a graphical illustration showing how any CDF starts from level 0 and rises to level 1, after which it continues to infinity.

📝 Lecture Summary

Third property of Cumulative Distribution Function and its Proof

Theorem 1: Let X be a random variable with cumulative distribution function F(x). Then, (c) lim x→∞ F(x) = 1 (the upper limit of F is 1).

Proof: We know that F(x) = P(X < x), so obviously F(∞) = P(X < ∞) is a sure event. Therefore, F(∞) = 1, hence lim x→∞ F(x) = 1.

🔑 Definition — Cumulative Distribution Function (CDF): F(x) = P(X < x), the probability that the random variable X takes a value less than x.

📐 Formula: lim x→∞ F(x) = 1 → As x increases without bound, the cumulative probability approaches certainty (1), because all possible values of X are less than infinity.

📌 Example (Graphical Interpretation): The F(x) of any random variable will always start from level 0 and rise up to level 1. Subsequent to that point 1, it goes up to infinity. This means the CDF begins at 0 for very small x values, increases as x increases, and eventually reaches and stays at 1 for sufficiently large x values, indicating all probability has been accumulated.

💡 Why this matters: This property ensures that the total probability under any probability distribution sums to exactly 1, which is a fundamental requirement for any valid probability distribution.

⭐ Key Takeaways

The third property of the cumulative distribution function states that the limit of F(x) as x approaches infinity equals 1, meaning the upper limit of any CDF is always 1. This is proven logically because F(∞) = P(X < ∞) represents a sure event, and the probability of a sure event is 1. Graphically, every CDF starts at level 0 and rises to level 1, after which it extends to infinity, demonstrating that all probability mass is captured by the time x reaches sufficiently large values. Students must remember that this property, along with the first two properties (F(x) is non-decreasing and lim x→-∞ F(x) = 0), completes the essential characteristics of any valid cumulative distribution function.

🧠 Quick Revision Questions

1. What is the third property of the cumulative distribution function?
2. Why does F(∞) equal 1?
3. What does the graphical representation of a CDF look like regarding its starting and ending levels?
4. What is the relationship between F(∞) and the concept of a sure event?
5. How does the third property of CDF relate to the total probability of a random variable?

📘 Lecture 21 — Fourth property of Cumulative Distribution Function and its Proof

📖 Overview: This lecture completes the discussion of the four properties of the Cumulative Distribution Function (CDF) by proving the fourth property — right continuity. Understanding why CDFs are right continuous is essential for working with probability distributions, especially when dealing with discrete random variables and points of discontinuity.

🗂️ Topics Covered

The lecture presents Theorem 1 part (d) regarding the right continuity of the Cumulative Distribution Function. It provides an explanation using the Discrete Uniform distribution as an example with a fair die toss, illustrates the concept through a graphical representation of a continuous CDF approaching a discontinuity point from the right, and presents the formal proof using a decreasing sequence of sets and the continuity theorem for probability measures.

📝 Lecture Summary

Fourth property of Cumulative Distribution Function and its Proof

Theorem 1: Let X be a random variable with cumulative distribution function F(x). Then (d) lim_(x↓x₀) F(x) = F(x₀) (F is right continuous).

Explanation: In case of a discrete distribution, we take an example of a Discrete Uniform distribution.

• Toss a fair die (1,2,3,4,5,6).
• The probability of getting a 1 or 2 or 3 or 4 or 5 or 6 are equally likely to occur.

The table shows:

Consider a CDF that is continuous everywhere other than at the point x₀ (ref. CDF's that have a few points of discontinuity). As x tends to x₀ from the R.H.S (right-hand side), the ordinate F(x) tends to the ordinate F(x₀).

💡 Why this matters: The right continuity property means that at any point where the CDF has a jump, the function value equals the limit from the right. This distinguishes CDFs from other cumulative functions and has important implications for how probabilities are assigned at individual points.

Part (d): Formal Proof

Let {xₙ} be any sequence of real numbers such that xₙ ↓ x₀. Let Cₙ = {X ≤ xₙ}. Then the sequence of sets {Cₙ} is decreasing and

⋂ₙ₌₁^∞ Cₙ = {X ≤ x₀}

🔑 Definition — Right Continuous: A function F is right continuous at a point x₀ if the limit of F(x) as x approaches x₀ from values greater than x₀ equals F(x₀). That is, lim_(x↓x₀) F(x) = F(x₀).

📐 Formula: lim_(x↓x₀) F(x) = F(x₀) → As we approach x₀ from the right side, the value of the CDF approaches the actual value of the CDF at x₀.

📌 Example: Using the fair die distribution, consider x₀ = 1. The limit as x approaches 1 from the right is F(1) = 1/6. For the discrete uniform distribution, F(1) = P(X ≤ 1) = P(X = 1) = 1/6. Approaching from the right, we never take values below 1, so the limit equals exactly 1/6.

Hence, by Theorem 3.6 (which states that for a decreasing sequence of sets, the probability of the intersection equals the limit of the probabilities), we have:

lim_(n→∞) F(xₙ) = F(x₀)

which is the desired result that the CDF is right continuous.

⭐ Key Takeaways

The fourth fundamental property of a Cumulative Distribution Function is right continuity, meaning that at any point x₀, the limit of F(x) as x approaches x₀ from the right equals F(x₀). The proof uses a decreasing sequence of sets {X ≤ xₙ} whose intersection is {X ≤ x₀} and applies the continuity theorem for probability measures. For discrete distributions like the uniform die toss, right continuity holds even at points of discontinuity, meaning the CDF takes the value at the point itself rather than the limit from the left. The formal proof requires considering any decreasing sequence converging to x₀ from the right and showing the corresponding set probabilities converge. This completes the set of four properties that characterize any valid cumulative distribution function.

🧠 Quick Revision Questions

1. State the fourth property of the Cumulative Distribution Function as given in Theorem 1 part (d).
2. In the proof of right continuity, what type of sequence {xₙ} is considered, and what does xₙ ↓ x₀ mean?
3. What is the relationship between the sequence of sets Cₙ = {X ≤ xₙ} and the set {X ≤ x₀} as n→∞?
4. Which theorem from probability theory is used to complete the proof of right continuity?
5. For a fair die toss, what is the value of F(2) and why does right continuity hold at x=2?

📘 Lecture 22 — Evaluating Probabilities using CDF

📖 Overview: This lecture introduces the fundamental theorem for computing probabilities using the Cumulative Distribution Function (CDF). It demonstrates how the CDF can be used to find probabilities for intervals of a random variable (X), replacing the need to work directly with probability density functions in many cases.

🗂️ Topics Covered

The lecture covers Theorem 1, which states that for a random variable (X) with CDF (F_X(x)), the probability (P[a < X < b]) equals (F_X(b) - F_X(a)). It provides a formal proof using the addition theorem of probability on mutually exclusive intervals. Finally, a numerical example is worked out using the exponential distribution to illustrate the application of the theorem.

📝 Lecture Summary

Theorem 1: Evaluating (P[a < X < b]) using CDF

Let (X) be a random variable with the CDF (F_X(x)). Then, for (a < b): [ P[a < X < b] = F_X(b) - F_X(a) ]

Proof: The interval ({-\infty < X < b}) can be written as the union of two mutually exclusive intervals: [ {-\infty < X < b} = {-\infty < X < a} \cup {a < X < b} ] Since these two intervals are mutually exclusive (they do not overlap), we apply the addition theorem of probability, which states that the probability of the union of mutually exclusive events equals the sum of their probabilities: [ P{-\infty < X < b} = P{-\infty < X < a} + P{a < X < b} ] Rearranging the terms yields: [ P{a < X < b} = P{-\infty < X < b} - P{-\infty < X < a} ] By definition of the CDF, (F_X(x) = P{-\infty < X < x}). Therefore: [ P[a < X < b] = F_X(b) - F_X(a) ]

💡 Why this matters: This theorem simplifies probability calculations dramatically. Instead of integrating a probability density function over an interval, you can simply take the difference of two CDF values — a much faster and more practical approach.

🔑 Definition — Mutually Exclusive Events: Two events are mutually exclusive if they cannot occur simultaneously. Their intersection is empty, so the probability of their union is simply the sum of their individual probabilities.

📐 Formula: (P[a < X < b] = F_X(b) - F_X(a)) → The probability that a random variable (X) falls in the interval ((a, b)) equals the CDF evaluated at (b) minus the CDF evaluated at (a).

Numerical Example

Find (P(1 < X < 2)) where (X) follows an exponential distribution with parameter (\theta = 2).

Step 1: Recall the CDF of an exponential distribution with parameter (\theta): [ F(x) = 1 - e^{-x/\theta}, \quad x \geq 0 ] Here (\theta = 2), so: [ F(x) = 1 - e^{-x/2} ]

Step 2: Apply the theorem: [ P(1 < X < 2) = F(2) - F(1) ]

Step 3: Calculate (F(2)): [ F(2) = 1 - e^{-2/2} = 1 - e^{-1} = 1 - \frac{1}{e} ]

Step 4: Calculate (F(1)): [ F(1) = 1 - e^{-1/2} = 1 - \frac{1}{\sqrt{e}} ]

Step 5: Compute the difference: [ P(1 < X < 2) = (1 - e^{-1}) - (1 - e^{-1/2}) = e^{-1/2} - e^{-1} ]

Step 6: Evaluate numerically using (e \approx 2.71828):

• (e^{-1/2} = \frac{1}{\sqrt{e}} = \frac{1}{\sqrt{2.71828}} = \frac{1}{1.64872} = 0.6065)
• (e^{-1} = \frac{1}{e} = \frac{1}{2.71828} = 0.3679)

Step 7: Final answer: [ P(1 < X < 2) = 0.6065 - 0.3679 = 0.2386 = 23.86% ]

📌 Example: For an exponential random variable with (\theta = 2), the probability that (X) lies between 1 and 2 is approximately 23.86%. This is found by computing (F(2) - F(1) = e^{-1/2} - e^{-1} \approx 0.2386).

⭐ Key Takeaways

The most critical concepts from this lecture are: (1) The probability that a continuous random variable (X) falls in any interval ((a, b)) can be computed directly from its CDF using the formula (P[a < X < b] = F_X(b) - F_X(a)). (2) This theorem is proved by expressing the interval as a union of mutually exclusive events and applying the addition theorem of probability. (3) The exponential distribution CDF, (F(x) = 1 - e^{-x/\theta}), was used in the example to demonstrate the method. (4) Numerical evaluation requires careful substitution and use of the exponential function. (5) This approach is much simpler than integrating the PDF, making CDFs a powerful tool for probability calculations.

🧠 Quick Revision Questions

1. State Theorem 1 for evaluating probabilities using the CDF.
2. What is the key property of the two intervals that allows the addition theorem to be applied in the proof?
3. In the example, what is the formula for the exponential CDF when (\theta = 2)?
4. What is the numerical value (to four decimal places) of (e^{-1/2}) and (e^{-1})?
5. If (F_X(3) = 0.8) and (F_X(1) = 0.2), what is (P(1 < X < 3))?