📘 Lecture 23 — Graphical Representation of the Distribution Function of a Discrete Random Variable
📖 Overview: This lecture explores the distribution function (cumulative distribution function) for discrete random variables, including its graphical representation as a step function. It then introduces the concept of mathematical expectation, covering the mean, variance, moments, and key properties of expected values — all essential for characterizing probability distributions.
🗂️ Topics Covered
The lecture begins with the definition and interpretation of the distribution function for discrete random variables, illustrated through the example of tossing three coins. It then presents another example involving defective clock radios. Following this, the concept of mathematical expectation is introduced, along with the expectation of functions of a random variable, variance, moments about the origin and about the mean, moment ratios for skewness and kurtosis, and finally, the properties of mathematical expectation.
📝 Lecture Summary
Graphical Representation of the Distribution Function of a Discrete Random Variable
The distribution function of a random variable X, denoted by F(x), is defined as F(x) = P(X ≤ x). It gives the probability that X takes a value less than or equal to a specified value x. It is also called the cumulative distribution function (cdf).
🔑 Definition — Distribution Function (F(x)): F(x) = P(X ≤ x), the cumulative probability of X from its smallest value up to x.
📌 Example: Find the probability distribution and distribution function for the number of heads when 3 balanced coins are tossed. The sample space S = {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}. Let X = number of heads. The probabilities are: f(0)=1/8, f(1)=3/8, f(2)=3/8, f(3)=1/8. The cumulative probabilities are: F(0)=1/8, F(1)=4/8, F(2)=7/8, F(3)=1. The distribution function is: F(x) = 0, for x < 0 F(x) = 1/8, for 0 ≤ x < 1 F(x) = 4/8, for 1 ≤ x < 2 F(x) = 7/8, for 2 ≤ x < 3 F(x) = 1, for x ≥ 3
The graph of the distribution function resembles a staircase and is known as a step function or jump function. For this example, the probability of at most one head is 4/8, and at most two heads is 7/8. 💡 Why this matters: The distribution function allows us to compute probabilities for ranges of values, not just individual points, which is fundamental for real-world decision-making.
📌 Example: A store places its last 15 clock radios in a clearance sale; 5 are defective. A customer tests 3 different radios selected at random. Let X = number of defective radios in the sample. The probability distribution is: P(X=0)=0.26, P(X=1)=0.49, P(X=2)=0.22, P(X=3)=0.02. The distribution function is: F(0)=0.26, F(1)=0.75, F(2)=0.97, F(3)≈1. The probability that the sample contains at most one defective radio is 0.75, and at most two defective radios is 0.97.
Mathematical Expectation
Let a discrete random variable X have possible values x₁, x₂, …, xₙ with corresponding probabilities f(x₁), f(x₂), …, f(xₙ) such that Σf(xᵢ)=1. The mathematical expectation or expected value of X, denoted by E(X), is defined as E(X) = x₁f(x₁) + x₂f(x₂) + … + xₙf(xₙ) = Σᵢ₌₁ⁿ xᵢ f(xᵢ). E(X) is also called the mean of X and is usually denoted by μ. It is a weighted mean of the variable’s possible values, each weighted by its probability. E(X) represents the average value of X over a very large number of trials.
🔑 Definition — Mathematical Expectation E(X): E(X) = Σ xᵢ f(xᵢ), the weighted average of all possible values of X, weighted by their probabilities.
📐 Formula: E(X) = Σᵢ xᵢ f(xᵢ) → The sum of each value of X multiplied by its probability.
📌 Example: An umbrella salesman earns $30 per day if it rains and loses $6 per day if it is fair. The probability of rain is 0.3. Let X = dollars earned. Possible values: 30 (rain, P=0.3) and -6 (no rain, P=0.7). E(X) = 30(0.3) + (-6)(0.7) = 9.0 - 4.2 = $4.80 per day. On average, the salesman expects to earn $4.80 per day.
Mean, Variance and Moments of a Discrete Probability Distribution
Let H(X) be a function of the random variable X. Then H(X) is also a random variable. The expected value of the function H(X) is E[H(X)] = Σ H(xᵢ) f(xᵢ). If H(X) = (X - μ)², then E(X - μ)² = Σ (xᵢ - μ)² f(xᵢ). This is called the variance, denoted by Var(X) or σ². The shortcut formula for variance is σ² = E(X²) - [E(X)]². The positive square root of the variance is the standard deviation.
If H(X) = Xᵏ, then E(Xᵏ) = Σ xᵢᵏ f(xᵢ) is called the kth moment about the origin, denoted by μ'ₖ. If H(X) = (X - μ)ᵏ, then μₖ = E(X - μ)ᵏ = Σ (xᵢ - μ)ᵏ f(xᵢ) is the kth moment about the mean.
Skewness is measured by β₁ = μ₃² / μ₂³, and kurtosis by β₂ = μ₄ / μ₂². These moment-ratios help determine skewness and kurtosis of a probability distribution.
🔑 Definition — Variance (σ²): σ² = E(X - μ)² = Σ (xᵢ - μ)² f(xᵢ), a measure of dispersion of the probability distribution.
📐 Formula: σ² = E(X²) - [E(X)]² → Variance equals the expected value of X² minus the square of the expected value of X.
Properties of Expected Values
-
If c is a constant, then E(c) = c. The expected value of a constant is the constant itself.
-
If X is a discrete random variable and a and b are constants, then E(aX + b) = a E(X) + b.
🔑 Definition — Linearity of Expectation: For constants a and b, E(aX + b) = a E(X) + b.
📌 Example: For three fair coins, X = number of heads, with distribution: P(0)=1/8, P(1)=3/8, P(2)=3/8, P(3)=1/8. E(X) = 0(1/8) + 1(3/8) + 2(3/8) + 3(1/8) = 12/8 = 1.5. Now find E(2X+3): E(2X+3) = 2E(X) + 3 = 2(1.5) + 3 = 3 + 3 = 6. Verification: For x=0: 2(0)+3=3, P=1/8 → 3/8; x=1: 5, P=3/8 → 15/8; x=2: 7, P=3/8 → 21/8; x=3: 9, P=1/8 → 9/8. Sum = (3+15+21+9)/8 = 48/8 = 6. ✓
⭐ Key Takeaways
The distribution function F(x) for a discrete random variable is a step function that gives cumulative probabilities, and its graph visually shows jumps at each possible value of X. Mathematical expectation E(X) is a weighted average representing the long-run average value of a random variable. Variance σ² = E(X²) - [E(X)]² measures spread, while moments about the origin and about the mean help characterize distributions. The linearity property E(aX + b) = aE(X) + b is crucial for simplifying expectation calculations.
🧠 Quick Revision Questions
- For the three-coin toss example, what is the probability of getting at most one head?
- How would you interpret F(2) = 7/8 in the context of the three-coin toss problem?
- A salesman earns $30 if it rains (P=0.3) and loses $6 if it doesn't (P=0.7). What is his expected daily earnings?
- Using the shortcut formula, how would you compute the variance of a discrete random variable?
- If E(X) = 1.5, what is E(2X + 3) and which property of expectation justifies your answer?
📘 Lecture 24 — Chebychev’s Inequality and Concept of Continuous Probability Distribution
📖 Overview: This lecture introduces Chebychev’s Inequality, a fundamental theorem that provides a guaranteed lower bound for the probability that a random variable falls within a specified number of standard deviations from its mean, applicable to any distribution with finite mean and variance. The lecture then transitions to continuous random variables, defining their probability density functions, distribution functions, and computing probabilities through integration, laying the groundwork for statistical inference.
🗂️ Topics Covered
The lecture covers Chebychev’s Inequality with examples using discrete probability distributions, including its application to business ventures. It then introduces the concept of continuous random variables and probability density functions (p.d.f.), their properties, and the distribution function. The lecture concludes with an example of computing probabilities, the distribution function, and conditional probabilities for a continuous distribution.
📝 Lecture Summary
Chebychev’s Inequality
If X is a random variable having mean μ and variance σ² > 0, and k is any positive constant, then the probability that a value of X falls within k standard deviations of the mean is at least 1 – 1/k². Formally: P(μ – kσ < X < μ + kσ) ≥ 1 – 1/k². Alternatively, the probability of observing a value that differs from μ by k or more standard deviations cannot exceed 1/k². This inequality, due to Russian mathematician P.L. Chebychev (1821–1894), holds for all probability distributions having finite mean and variance and provides a means of understanding how the standard deviation measures variability about the mean.
🔑 Definition — Chebychev’s Inequality: For any random variable X with finite mean μ and variance σ²>0, and any positive constant k, P(|X – μ| ≥ kσ) ≤ 1/k², or equivalently P(|X – μ| < kσ) ≥ 1 – 1/k². 📌 Example: For a flower petal distribution with μ = 5.925 and σ = 1.3, using k=2, Chebychev’s inequality states that at least 1 – 1/2² = 1 – 1/4 = 0.75 of the probability lies between μ – 2σ = 3.3 and μ + 2σ = 8.5. The actual probability P(3.3 < X < 8.5) = 0.10 + 0.20 + 0.30 + 0.25 + 0.075 = 0.925, which is greater than 0.75.
📌 Example — Business Ventures: For a distribution of successful ventures out of five with μ = 3.50 and σ = 1.02, the interval μ ± 2σ = (1.46, 5.54). Chebychev’s rule guarantees at least 75% of observations fall in this interval. The actual probability P(2 ≤ X ≤ 5) = 0.132 + 0.309 + 0.360 + 0.168 = 0.969, consistent with both Chebychev’s rule and the Empirical Rule. Fewer than two successful ventures (x=0 or x=1) is unlikely, with exact probability P(x<2) = 0.002 + 0.029 = 0.031.
💡 Why this matters: Chebychev’s Inequality and the Empirical Rule assist in understanding how much data lies between the mean minus a certain number of standard deviations and mean plus that same number of standard deviations, providing a basis for statistical inference.
Concept of Continuous Probability Distribution
A continuous random variable can assume every possible value in an interval [a, b], where a and b may be –∞ and +∞ respectively. The function f(x) is called the probability density function (p.d.f.) of X. A continuous probability distribution has the following properties: i) f(x) ≥ 0, for all x ii) ∫_{-∞}^{∞} f(x) dx = 1 (total area under the curve is 1) iii) The probability that X takes on a value in the interval [c, d] is given by P(c < X < d) = ∫_c^d f(x) dx, which is the area under the curve between X=c and X=d.
Since P(X = k) = ∫_k^k f(x) dx = 0, the probability of a continuous random variable taking any particular value is always zero. Therefore, for a continuous random variable, P(c < X < d), P(c ≤ X < d), P(c < X ≤ d), and P(c ≤ X ≤ d) are regarded as the same.
🔑 Definition — Continuous Random Variable: A random variable X is continuous if its distribution function F(x) is continuous and differentiable everywhere except at isolated points in the given range. The graph of F(x) has no jumps or steps but is a continuous function for all x-values. 📐 Relationship: f(x) = dF(x)/dx, and F(x) = ∫_{-∞}^x f(t) dt, for all x.
Mathematical Expectation, Variance & Moments of a Continuous Probability Distribution
For a continuous random variable X with p.d.f. f(x), the expected value (mean) is defined as E(X) = ∫_{-∞}^{∞} x f(x) dx. Properties of mathematical expectation, such as E(aX + b) = a E(X) + b, hold for continuous distributions. Moments about the origin (μ’_r) are given by μ’r = E(X^r) = ∫{-∞}^{∞} x^r f(x) dx. Moments about the mean (μ_r) are computed using relationships between moments about the origin, from which moment-ratios (β₁ and β₂) are derived.
📌 Example — Expected Value: For f(x) = 2(1 – x), 0 < x < 1, E(X) = 2∫_0^1 x(1 – x) dx = 2[x²/2 – x³/3]₀¹ = 2(1/2 – 1/3) = 1/3. For a=3 and b=5, E(3X+5) = 3(1/3) + 5 = 6, verified by direct integration.
📌 Example — Moments: For f(x) = (3/4)x(2 – x), 0 ≤ x ≤ 2, the first four moments about the origin are: μ’₁ = 1, μ’₂ = 6/5, μ’₃ = 8/5, μ’₄ = 16/7. Moments about the mean: μ₁ = 0, μ₂ = 1/5, μ₃ = 0, μ₄ = 3/35. Moment-ratios: β₁ = μ₃²/μ₂³ = 0 (absolutely symmetric), β₂ = μ₄/μ₂² = 2.14 (platykurtic).
⭐ Key Takeaways
Chebychev’s Inequality provides a universal bound: for any distribution with finite variance, at least 1 – 1/k² of the data lies within k standard deviations of the mean. It works for all distributions, while the Empirical Rule applies specifically to mound-shaped distributions. Continuous random variables have probability density functions where probabilities are areas under the curve, and the probability of any single point is zero. The expected value and moments for continuous distributions are computed using integration, and properties of expectation carry over from discrete cases. The concept of joint (bivariate) probability distributions introduces marginal and conditional probability functions, with independence defined as the joint probability equaling the product of marginal probabilities.
🧠 Quick Revision Questions
- What does Chebychev’s Inequality state about the probability that a random variable falls within k standard deviations of its mean?
- For a continuous random variable, why is P(X = k) always zero, and what are the implications for interval probabilities?
- A p.d.f. is given by f(x) = kx for 0 ≤ x ≤ 2. Find the value of k that makes this a valid density function.
- For the same p.d.f. f(x) = x/2, 0 ≤ x ≤ 2, compute P(X > 1) and find the distribution function F(x).
- A joint probability distribution lists f(0,1) = 6/28 and the marginal g(0) = 10/28 and h(1) = 12/28. Are X and Y independent at these values? Why or why not?
📘 Lecture 26 — STA301 Statistics and Probability
📖 Overview: This lecture completes the study of bivariate probability distributions by working through a detailed discrete example involving drawing balls from an urn. It then introduces continuous bivariate distributions, including joint density functions, marginal densities, conditional densities, and the concept of statistical independence. Finally, important properties of expected values for bivariate distributions are presented.
🗂️ Topics Covered
The lecture works through a complete discrete bivariate example finding joint, marginal, and conditional probability distributions and testing independence. It then introduces continuous bivariate distributions with definitions of joint, marginal, and conditional density functions, followed by a worked example. The lecture concludes with two key properties of expected values for sums and products of random variables.
📝 Lecture Summary
BIVARIATE Probability Distributions (Discrete and Continuous)
The lecture begins by completing a discrete example involving drawing 2 balls from an urn containing 3 black, 2 red, and 3 green balls. Let X be the number of black balls and Y be the number of red balls selected.
Using the rule of combinations with the classical definition of probability, the following joint probability distribution table is obtained:
| X | Y | 0 | 1 | 2 | P(X=x_i) g(x) |
|---|---|---|---|---|---|
| 0 | 3/28 | 6/28 | 1/28 | 10/28 | |
| 1 | 9/28 | 6/28 | 0 | 15/28 | |
| 2 | 3/28 | 0 | 0 | 3/28 | |
| P(Y=y_j) h(y) | 15/28 | 12/28 | 1/28 | 1 |
The joint p.d. formula is: f(x,y) = [(3 choose x)(2 choose y)(3 choose 2-x-y)] / (8 choose 2), where x=0,1,2; y=0,1,2; 0 ≤ x+y ≤ 2.
To compute P(X+Y < 1), we identify cells where x+y < 1: (0,0), (0,1), and (1,0). Therefore, P(X+Y < 1) = f(0,0) + f(0,1) + f(1,0) = 3/28 + 6/28 + 9/28 = 18/28 = 9/14.
The marginal p.d. g(x) and h(y) are obtained by summing probabilities across rows and columns respectively:
- g(x): x=0 → 10/28, x=1 → 15/28, x=2 → 3/28
- h(y): y=0 → 15/28, y=1 → 12/28, y=2 → 1/28
The conditional p.d. f(x|1) = P(X=x | Y=1) = f(x,1)/h(1). Since h(1)=12/28=3/7, we compute:
- f(0|1) = (7/3)×(6/28) = 1/2
- f(1|1) = (7/3)×(6/28) = 1/2
- f(2|1) = (7/3)×0 = 0
Hence, the conditional distribution of X given Y=1 is: x=0 → 1/2, x=1 → 1/2, x=2 → 0.
To test independence, check if f(0,1) = g(0)h(1). Since f(0,1)=6/28, g(0)=10/28, and h(1)=12/28, we see that 6/28 ≠ (10/28)×(12/28). Therefore, X and Y are NOT statistically independent.
CONTINUOUS BIVARIATE DISTRIBUTIONS
The bivariate probability density function of continuous r.v.’s X and Y is an integral function f(x,y) satisfying:
🔑 Definition — Bivariate continuous joint p.d.f.: f(x,y) is a function satisfying:
- i) f(x,y) ≥ 0 for all (x,y)
- ii) ∫∫ f(x,y) dx dy = 1 over the entire XY-plane
- iii) P(a ≤ X ≤ b, c ≤ Y ≤ d) = ∫ₐᵇ ∫_cᵈ f(x,y) dy dx
The probability function f(x,y) gives a SURFACE, and computing P(x₁ < X < x₂, y₁ < Y < y₂) means computing the VOLUME under the surface over that rectangular region.
The marginal p.d.f. of continuous r.v. X is: g(x) = ∫{-∞}^{∞} f(x,y) dy, and the marginal p.d.f. of Y is: h(y) = ∫{-∞}^{∞} f(x,y) dx.
The conditional p.d.f. of X given Y=y is: f(x|y) = f(x,y)/h(y), where h(y) > 0. Similarly, f(y|x) = f(x,y)/g(x), where g(x) > 0.
Two continuous r.v.’s X and Y are statistically independent if and only if their joint density can be factorized as: f(x,y) = g(x)h(y) for all possible values of X and Y.
📐 Formula — Continuous bivariate independence condition: f(x,y) = g(x)h(y) for all x, y
An example is worked through with joint p.d.f. f(x,y) = (1/8)(6-x-y), 0 ≤ x ≤ 2, 2 ≤ y ≤ 4, and 0 elsewhere.
a) Verification as a joint density: ∫₀² ∫₂⁴ (1/8)(6-x-y) dy dx = (1/8)[12-4] = 1. Also f(x,y) > 0 in the region. So it satisfies all properties.
b) P(X ≤ 3/2, Y ≤ 5/2) = ∫₀^{3/2} ∫₂^{5/2} (1/8)(6-x-y) dy dx = (1/8)[(15/2)x - x²]_0^{3/2} = (1/8)(45/4 - 9/4) = (1/8)(36/4) = 9/32.
c) Marginal p.d.f. of X: g(x) = ∫₂⁴ (1/8)(6-x-y) dy = (1/4)(3-x), 0 ≤ x ≤ 2. Marginal p.d.f. of Y: h(y) = ∫₀² (1/8)(6-x-y) dx = (1/4)(5-y), 2 ≤ y ≤ 4.
d) Conditional p.d.f. of X given Y=y: f(x|y) = f(x,y)/h(y) = [(1/8)(6-x-y)] / [(1/4)(5-y)] = (6-x-y)/(2(5-y)), 0 ≤ x ≤ 2. Conditional p.d.f. of Y given X=x: f(y|x) = f(x,y)/g(x) = [(1/8)(6-x-y)] / [(1/4)(3-x)] = (6-x-y)/(2(3-x)), 2 ≤ y ≤ 4.
Properties of Expected Values in the case of Bivariate Probability Distributions
🔑 Definition — Property No. 1: E(X+Y) = E(X) + E(Y). This also holds for differences: E(X-Y) = E(X) - E(Y)
🔑 Definition — Property No. 2: If X and Y are independent r.v.’s, then E(XY) = E(X)E(Y)
These properties are valid for both discrete and continuous random variables.
📌 Example: Given the joint distribution:
| x \ y | 2 | 4 |
|---|---|---|
| 1 | 0.10 | 0.15 |
| 3 | 0.20 | 0.30 |
| 5 | 0.10 | 0.15 |
First find marginal distributions: g(x): x=1→0.25, x=3→0.50, x=5→0.25; h(y): y=2→0.40, y=4→0.60.
E(X) = 1×0.25 + 3×0.50 + 5×0.25 = 0.25 + 1.50 + 1.25 = 3.0 E(Y) = 2×0.40 + 4×0.60 = 0.80 + 2.40 = 3.2 E(X+Y) = E(X) + E(Y) = 3.0 + 3.2 = 6.2
Alternatively, E(X+Y) = ΣΣ(x_i + y_j) f(x_i, y_j) over all i, j. E(XY) = ΣΣ(x_i y_j) f(x_i, y_j) over all i, j.
⭐ Key Takeaways
The joint probability distribution for discrete random variables must sum to 1 over all cells, and the marginal distributions are obtained by summing across rows or columns. Conditional probability distributions are found by dividing joint probabilities by marginal probabilities. For continuous bivariate distributions, volumes under the surface represent probabilities, and marginal densities are found by integrating out the other variable. Two random variables are statistically independent only if their joint distribution equals the product of their marginals for all values. The expected value of a sum always equals the sum of expected values, but the expected value of a product equals the product of expected values only when the variables are independent.
🧠 Quick Revision Questions
- In the urn example with black, red, and green balls, what is the value of P(X=0, Y=2)?
- What are the four properties that a continuous bivariate joint density function must satisfy?
- For the example f(x,y) = (1/8)(6-x-y) with 0≤x≤2, 2≤y≤4, what is P(X>1, Y<3)?
- When computing the conditional p.d.f. f(x|y), what condition must the marginal p.d.f. h(y) satisfy?
- If E(X) = 3.0 and E(Y) = 3.2, what is E(2X - 3Y + 5)?
📘 Lecture 27 — Properties of Expected Values in the case of Bivariate Probability Distributions (Detailed discussion) — Covariance & Correlation — Some Well-known Discrete Probability Distributions: — Discrete Uniform Distribution — An Introduction to the Binomial Distribution
📖 Overview: This lecture explores the key properties of mathematical expectation for bivariate probability distributions, establishing foundational rules for expected values of sums and products of random variables. It introduces covariance and correlation as measures of the relationship between two variables, and begins the study of important discrete probability distributions, specifically the discrete uniform and binomial distributions.
🗂️ Topics Covered
The lecture covers the properties of expected values for sums and products of random variables in both discrete and continuous bivariate distributions, verified through detailed examples. It then introduces covariance and the correlation coefficient as measures of linear association, followed by a discussion of the discrete uniform distribution illustrated with a die-tossing example and a lottery scenario. Finally, the lecture provides an introduction to the binomial distribution and the fundamental properties of a binomial experiment.
📝 Lecture Summary
Properties of Expected Values in the case of Bivariate Probability Distributions (Detailed discussion)
This section examines two key properties of mathematical expectation for bivariate distributions. First, the expected value of the sum of two random variables equals the sum of their individual expected values: E(X + Y) = E(X) + E(Y). Second, for independent random variables, the expected value of the product equals the product of their expected values: E(XY) = E(X)E(Y).
🔑 Definition — Mathematical Expectation: For discrete r.v.’s, the expected value of a function h(x,y) is E[h(X,Y)] = ΣᵢΣⱼ h(xᵢ,yⱼ) f(xᵢ,yⱼ).
📐 Formula: E(X + Y) = E(X) + E(Y) → The expected value of the sum is the sum of the expected values.
📐 Formula: E(XY) = E(X)E(Y) → Only holds when X and Y are independent.
📌 Example: For the joint p.d. where X takes values 2 and 4, and Y takes values 1, 3, and 5:
- E(X) = 2 × 0.40 + 4 × 0.60 = 3.2
- E(Y) = 1 × 0.25 + 3 × 0.50 + 5 × 0.25 = 3.0
- E(X + Y) = (2+1)(0.10) + (2+3)(0.20) + (2+5)(0.10) + (4+1)(0.15) + (4+3)(0.30) + (4+5)(0.15) = 6.20 = E(X) + E(Y)
- E(XY) = (2×1)(0.10) + (2×3)(0.20) + (2×5)(0.10) + (4×1)(0.15) + (4×3)(0.30) + (4×5)(0.15) = 9.6 = E(X)E(Y) → confirming independence.
📌 Example: For continuous r.v.’s with joint p.d.f. f(x,y) = x(1+3y²)/4, 0<x<2, 0<y<1:
- E(X) = 4/3, E(Y) = 5/8
- E(X + Y) = 47/24 = E(X) + E(Y) ✓
- E(XY) = 5/6 = E(X)E(Y) ✓
💡 Why this matters: These properties simplify computation and help determine independence between variables.
Covariance & Correlation
This section introduces measures of association between two random variables. Covariance quantifies the direction of the linear relationship, while the correlation coefficient provides a standardized measure of linear correlation strength.
🔑 Definition — Covariance: Cov(X,Y) = E{[X - E(X)][Y - E(Y)]} = E(XY) - E(X)E(Y), measuring how X and Y vary together.
📐 Formula: Cov(X,Y) = E(XY) - E(X)E(Y) → The short-cut formula for covariance.
📐 Formula: ρXY = Cov(X,Y) / (σXσY) → The correlation coefficient, ranging from -1 to 1.
📌 Example: For the joint p.d. with x=0,1,2 and y=0,1,2,3:
- E(X) = 1.10, E(Y) = 1.65
- Var(X) = 0.49, Var(Y) = 0.7275
- E(XY) = 1.90
- Cov(X,Y) = 1.90 - 1.10×1.65 = 0.085
- ρ = 0.085/√(0.49×0.7275) = 0.14 → weak positive linear correlation.
📌 Example: For f(x,y) = x² + xy/3, 0<x<1, 0<y<2:
- E(X) = 13/18, E(Y) = 10/9
- Var(X) = 73/1620, Var(Y) = 26/81
- Cov(X,Y) = -1/162
- ρ = -0.05 → very weak negative linear correlation, almost uncorrelated.
💡 Why this matters: Independent variables have zero covariance/correlation, but the converse is not true—zero correlation does not imply independence.
Some Well-known Discrete Probability Distributions: — Discrete Uniform Distribution
The discrete uniform distribution arises when each possible outcome of a random variable is equally likely to occur. It is characterized by all values having the same probability of occurrence.
🔑 Definition — Discrete Uniform Distribution: A probability distribution where the random variable X assumes each of its k values with equal probability. For a fair die, each value from 1 to 6 has probability 1/6.
📌 Example: Tossing a fair die: X represents the number of dots on the upper-most face. Each X-value from 1 to 6 occurs with probability 1/6. The mean lies at the exact center of the distribution, i.e., E(X) = 3.5.
📌 Example: Lottery with tickets numbered 0000 to 9999, where each number is equally likely to be selected. This reflects the discrete uniform distribution because all outcomes have the same probability of occurrence.
An Introduction to the Binomial Distribution
The binomial distribution is a fundamental discrete probability distribution discovered by James Bernoulli around 1700. It models the number of successes in a fixed number of independent trials with constant success probability.
🔑 Definition — Binomial Distribution: The probability distribution of the number of successes (X) in n independent Bernoulli trials, where each trial has constant success probability p and failure probability q = 1-p.
📌 Example: Tossing a fair coin 5 times, where X represents the number of heads. Key properties include:
- Every toss results in either a head (success) or tail (failure)
- The probability of heads p = 1/2 remains constant
- Each throw is independent
- The total number of tosses n = 5 is fixed in advance
📐 Properties of a Binomial Experiment:
- Every trial results in a success or a failure
- Successive trials are independent
- The probability of success, p, remains constant from trial to trial
- The number of trials, n, is fixed in advance
⭐ Key Takeaways
The expected value of the sum of two random variables always equals the sum of their individual expected values, regardless of independence. For independent random variables, the expected value of the product equals the product of the expected values. Covariance and correlation provide numerical measures of linear association, with zero values for independent variables, though zero correlation does not guarantee independence. The discrete uniform distribution applies when all outcomes are equally likely, such as with a fair die or a random lottery. The binomial distribution applies to experiments with fixed n independent trials, constant success probability p, and binary outcomes (success/failure).
🧠 Quick Revision Questions
- What is the relationship between E(X+Y), E(X), and E(Y) in a bivariate probability distribution?
- Under what condition does E(XY) equal E(X)E(Y)?
- What is the short-cut formula for covariance, and what does Cov(X,Y) = 0 imply about independence?
- Define the discrete uniform distribution and give a real-world example where it applies.
- List the four essential properties of a binomial experiment.
📘 Lecture 28 — Binomial Distribution, Fitting a Binomial Distribution to Real Data, An Introduction to the Hypergeometric Distribution
📖 Overview: This lecture explores the binomial distribution, a fundamental discrete probability distribution for modeling success/failure experiments. It covers its properties, parameters, mean and standard deviation, and shows how to fit a binomial distribution to real-world observed data. The lecture concludes with an introduction to the hypergeometric distribution, which applies when sampling is done without replacement and the probability of success changes on each trial.
🗂️ Topics Covered
The lecture begins by illustrating the binomial distribution using the example of tossing a fair coin five times, detailing its four essential properties and the formula for calculating probabilities. It then presents the formulas for the mean and standard deviation of a binomial distribution and discusses the distribution's shape (symmetrical vs. skewed). The second half of the lecture demonstrates the process of fitting a binomial distribution to real data from a loaded die, including calculating expected frequencies and comparing them to observed frequencies. The lecture concludes by introducing the hypergeometric distribution, its properties, and its probability formula, highlighting when it is used instead of the binomial distribution.
📝 Lecture Summary
Binomial Distribution
The binomial distribution is a very important discrete probability distribution. It applies to a binomial experiment which has four basic and vitally important properties: (1) every trial results in either a success or failure, (2) the probability of success (p) remains constant for all trials, (3) the trials are independent, and (4) the number of trials (n) is fixed in advance.
The probability distribution is given by the formula: P(X = x) = (n choose x) p^x q^(n-x)
Where: n = the total number of trials p = probability of success in each trial q = probability of failure in each trial (i.e., q = 1 - p) x = number of successes in n trials (x = 0, 1, 2, ..., n)
The binomial distribution has two parameters, n and p.
🔑 Definition — Binomial Distribution: A discrete probability distribution that models the number of successes in a fixed number of independent trials, each with the same probability of success.
📐 Formula: P(X = x) = (n chooses x) p^x q^(n-x) → The probability of getting exactly x successes in n trials equals the number of ways to choose x successes from n trials, multiplied by the probability of x successes and the probability of (n-x) failures.
📌 Example: Tossing a fair coin 5 times (n=5, p=0.5). Calculate P(X = 0 heads) = (5 choose 0) (1/2)^0 (1/2)^5 = 1 * 1 * 1/32 = 1/32. P(X = 1 head) = (5 choose 1) (1/2)^1 (1/2)^4 = 5 * (1/2) * (1/16) = 5/32. The complete distribution is: P(0)=1/32, P(1)=5/32, P(2)=10/32, P(3)=10/32, P(4)=5/32, P(5)=1/32.
💡 Why this matters: The formula allows us to calculate the probability of any number of successes in a binomial experiment without having to list all possible outcomes.
Mean and Standard Deviation of the Binomial Distribution
For a binomial distribution, there are simple formulas for the mean and standard deviation. It has been mathematically proved that:
For a binomial distribution: E(X) = np Var(X) = npq so that S.D.(X) = √(npq)
🔑 Definition — Mean of Binomial Distribution (μ): np → The average number of successes expected over many repetitions of the binomial experiment.
🔑 Definition — Standard Deviation of Binomial Distribution (σ): √(npq) → A measure of the spread or variability of the possible number of successes around the mean.
📐 Formulas: μ = np; σ = √(npq) → The mean is the product of trials and success probability; the standard deviation is the square root of the product of trials, success probability, and failure probability.
📌 Example: For the fair coin toss example (n=5, p=0.5, q=0.5): Mean = np = 5(0.5) = 2.5 S.D.(X) = √(5 * 0.5 * 0.5) = √(5/4) = 1.12 This means that if 5 fair coins are tossed an infinite number of times, we should expect to get 2.5 heads on average. The standard deviation of 1.12 gives a measure of the possible variability in the number of heads. Coefficient of Variation: C.V. = σ/μ * 100 = 1.12/2.5 * 100 = 44.8%
Shape of the Binomial Distribution
The binomial distribution is not always symmetrical. Its shape depends on the values of p and q:
- It is symmetrical only when p = q = ½.
- It is skewed to the right if p < q.
- It is skewed to the left if p > q.
The degree of skewness (or asymmetry) decreases as n increases.
🔑 Definition — Skewness: The asymmetry in a probability distribution where the tail on one side is longer or fatter than the other. For binomial, skewness depends on the relative values of p and q.
Fitting a Binomial Distribution to Real Data
When we have observed data from a binomial experiment, we can fit a theoretical binomial distribution to it. This involves estimating the parameters n and p from the data.
The procedure is:
- Determine n from the largest x-value in the data.
- Estimate p using the relationship μ = np, but since μ is unknown, we replace it with its estimate x̄ (sample mean). So, x̄ = np.
- Calculate the theoretical probabilities using the estimated binomial distribution.
- Multiply these probabilities by the total frequency to get expected frequencies.
- Compare observed frequencies with expected frequencies to assess the goodness of fit.
📌 Example: The data below was obtained by tossing a loaded die 5 times, noting the number of sixes:
| No. of Sixes (x) | 0 | 1 | 2 | 3 | 4 | 5 | Total |
|---|---|---|---|---|---|---|---|
| Frequency (fo) | 12 | 56 | 74 | 39 | 18 | 1 | 200 |
Solution:
- n = 5 (largest x-value)
- Calculate x̄: x̄ = Σ(fi xi) / Σfi = (0 + 56 + 148 + 117 + 72 + 5) / 200 = 398/200 = 1.99
- Using x̄ = np: 5p = 1.99 → p = 0.398
- The fitted binomial distribution is b(x; 5, 0.398)
- Calculate probabilities and expected frequencies:
| x | Probability Calculation | f(x) | Expected Frequency (200 * f(x)) |
|---|---|---|---|
| 0 | q^5 = (0.602)^5 | 0.07907 | 15.8 |
| 1 | 5 q^4 p = 5(0.602)^4(0.398) | 0.26136 | 52.5 |
| 2 | 10 q^3 p^2 = 10(0.602)^3(0.398)^2 | 0.34559 | 69.1 |
| 3 | 10 q^2 p^3 = 10(0.602)^2(0.398)^3 | 0.22847 | 45.7 |
| 4 | 5 q p^4 = 5(0.602)(0.398)^4 | 0.07553 | 15.1 |
| 5 | p^5 = (0.398)^5 | 0.00998 | 2.0 |
| Total | 1.00000 | 200.0 |
Comparing observed and expected frequencies shows not much discrepancy, indicating a reasonably good fit. The Chi-Square Test of Goodness of Fit is a formal method to determine if the fit is adequate.
💡 Why this matters: Fitting a theoretical distribution to real data allows us to model and analyze real-world phenomena using a known mathematical framework.
Real-Life Application from Industry
The binomial distribution has many real-life applications, such as quality control. When dealing with a success/failure situation, we are often dealing with a binomial experiment. Examples include proportion of smokers, literates, or males in a population.
📌 Example: A factory has a past record of 7% defective articles. A new law states there should not be more than 5% defective. The factory owner claims defectives have decreased. An inspector examines a sample of 20 items. What is the probability of finding 2 or more defective items (so a fine is imposed)?
Solution: This is a binomial experiment (n=20, p=0.07, q=0.93). P(X ≥ 2) = 1 - P(X < 2) = 1 - [P(X=0) + P(X=1)] P(X=0) = (20 choose 0)(0.07)^0(0.93)^20 = 1 * 1 * 0.234 = 0.234 P(X=1) = (20 choose 1)(0.07)^1(0.93)^19 = 20 * 0.07 * 0.353 = 0.4942 (correcting the lecture's calculation to 0.353 * 1.4? wait, let's do carefully) P(X=1) = 20 * 0.07 * (0.93)^19 ≈ 1.4 * 0.353 ≈ 0.494? Wait, 20*0.07 = 1.4. (0.93)^19 can be calculated, but lecture says 0.353 for P(X=1) and 0.234 for P(X=0). Let's use lecture's numbers. P(X=0) = 0.234 P(X=1) = 20 * 0.07 * 0.93^19 = 1.4 * 0.93^19. Lecture says this equals 0.353. So, P(X=1) = 0.353. P(X ≥ 2) = 1 - (0.234 + 0.353) = 1 - 0.587 = 0.413 = 41.3%
Hence, there is a substantial probability (more than 40%) that the inspector will find two or more defective articles, meaning a considerable chance the factory will be fined.
Hypergeometric Probability Distribution
There are many experiments where the condition of independence is violated and the probability of success does not remain constant for all trials. Such experiments are called hypergeometric experiments.
Properties of a Hypergeometric Experiment:
- The outcomes of each trial may be classified into one of two categories, success and failure.
- The probability of success changes on each trial.
- The successive trials are not independent.
- The experiment is repeated a fixed number of times.
The number of successes, X, in a hypergeometric experiment is called a hypergeometric random variable, and its probability distribution is the hypergeometric distribution.
The formula is: P(X = x) = ( (k choose x) * (N-k choose n-x) ) / (N choose n)
Where: N = number of units in the population n = number of units in the sample k = number of successes in the population
The hypergeometric probability distribution has three parameters: N, n, and k.
It is appropriate when:
- A random sample of size n is drawn WITHOUT REPLACEMENT from a finite population of N units.
- k of the units are of one kind (classified as success) and the remaining N-k of another kind (classified as failure).
🔑 Definition — Hypergeometric Distribution: A discrete probability distribution that models the number of successes in a sample drawn without replacement from a finite population containing a fixed number of successes.
📐 Formula: P(X = x) = [C(k, x) * C(N-k, n-x)] / C(N, n) → The probability of getting exactly x successes in a sample of size n is the number of ways to choose x successes from k successes multiplied by the number of ways to choose (n-x) failures from (N-k) failures, divided by the total number of ways to choose a sample of size n from the population of size N.
💡 Why this matters: The hypergeometric distribution is used when sampling is done without replacement, where the binomial distribution would not be appropriate because the probability of success changes with each draw.
⭐ Key Takeaways
- The binomial distribution is defined by two parameters, n (number of trials) and p (probability of success), and its formula P(X=x) = (n choose x) p^x q^(n-x) allows for calculating probabilities of any number of successes.
- For any binomial distribution, the mean is np and the standard deviation is √(npq), and its shape is symmetrical only when p=q=0.5, skewed right when p<q, and skewed left when p>q.
- To fit a binomial distribution to real data, estimate p using the sample mean (x̄ = np) and then calculate expected frequencies to compare with observed frequencies.
- The hypergeometric distribution is used instead of the binomial distribution when sampling is done without replacement from a finite population, making trials dependent and changing the probability of success.
- The hypergeometric probability formula uses three parameters (N, n, k) and is expressed as P(X=x) = [(C(k,x) * C(N-k, n-x)] / C(N,n).
🧠 Quick Revision Questions
- What are the four properties a random experiment must satisfy to be considered a binomial experiment?
- A binomial distribution has n=10 and p=0.3. Calculate its mean and standard deviation.
- Explain the procedure for fitting a binomial distribution to a given set of observed frequencies.
- In the loaded die example with n=5 and total frequency 200, why was the calculated value of p=0.398 considered evidence that the die was not fair?
- Under what specific conditions would you use the hypergeometric distribution instead of the binomial distribution?
📘 Lecture 29 — Hypergeometric Distribution, Poisson Distribution, Limiting Approximation to the Binomial, Poisson Process, Continuous Uniform Distribution
📖 Overview: This lecture completes the discussion of the hypergeometric probability distribution and introduces the Poisson distribution as both a limiting approximation to the binomial and a distribution in its own right. It also begins the discussion of continuous probability distributions with the uniform distribution. Understanding these distributions is essential for modeling real-world phenomena where events occur randomly over time or space.
🗂️ Topics Covered
The lecture covers hypergeometric distribution properties and its relationship to the binomial distribution, the Poisson distribution as a limiting approximation to the binomial when n is large and p is small, the Poisson process for random events over intervals, properties of the Poisson distribution including its mean-variance equality, fitting a Poisson distribution to real data, and the continuous uniform distribution with its mean, variance, and rectangular shape.