📘 Lecture 33 — Sampling Distribution of (continued) & Point Estimation
📖 Overview: This lecture continues the discussion of sampling distributions, specifically focusing on the sampling distribution of the difference between two sample means and the sampling distribution of the difference between two sample proportions. It then introduces the fundamental concepts of point estimation, including the distinction between an estimator and an estimate, and examines two desirable qualities of a good point estimator: unbiasedness and consistency.
🗂️ Topics Covered
The lecture covers a practical example applying the sampling distribution of the difference between two means, then introduces the sampling distribution of the difference between two proportions along with its properties and an example. The discussion shifts to point estimation, differentiating between an estimator and an estimate, and explains the concept of standard error. Finally, it details the desirable qualities of a good point estimator, specifically unbiasedness and consistency, with examples and formula derivations.
📝 Lecture Summary
Sampling Distribution of X̅₁ - X̅₂ (continued)
The lecture begins with a real-life application of the sampling distribution of the difference between two sample means, illustrating how to compute probabilities concerning differences in sample means from two independent populations.
🔑 Definition — Sampling Distribution of X̅₁ - X̅₂: The probability distribution of all possible differences between two independent sample means drawn from two populations.
📐 Formula: For large sample sizes (n₁, n₂ ≥ 30), the sampling distribution of X̅₁ - X̅₂ is approximately normal with:
- Mean: μₓ₁₋ₓ₂ = μ₁ - μ₂
- Standard Error: σₓ₁₋ₓ₂ = √(σ₁²/n₁ + σ₂²/n₂)
The standardized variable is: Z = [(X̅₁ - X̅₂) - (μ₁ - μ₂)] / √(σ₁²/n₁ + σ₂²/n₂)
📌 Example: Car batteries from company A have μ₁ = 4.3 years, σ₁ = 0.6 years. Company B batteries have μ₂ = 4.0 years, σ₂ = 0.4 years. Find the probability that a random sample of 49 batteries from A will have a mean life at least 0.5 years more than the mean life of a sample of 36 batteries from B.
Step 1: Compute the mean of the sampling distribution: μₓ₁₋ₓ₂ = 4.3 - 4.0 = 0.3 years
Step 2: Compute the standard error: σₓ₁₋ₓ₂ = √((0.6)²/49 + (0.4)²/36) = √(0.36/49 + 0.16/36) = √(0.00735 + 0.00444) = √0.01179 = 0.1086 years
Step 3: Compute the z-value for X̅₁ - X̅₂ = 0.5: z = (0.5 - 0.3) / 0.1086 = 0.2 / 0.1086 = 1.84
Step 4: Find the probability: P(X̅₁ - X̅₂ ≥ 0.5) = P(Z ≥ 1.84) = 0.5 - P(0 < Z < 1.84) = 0.5 - 0.4671 = 0.0329
The probability is 3.29%. Given the true difference in mean lifetimes is 0.3 years, there is only a 3.3% chance of observing a sample difference of at least 0.5 years. 💡 Why this matters: This demonstrates how to assess whether an observed difference between two sample means is statistically significant or could have occurred by random chance.
Sampling Distribution of the Differences Between Proportions
This section introduces the sampling distribution for comparing proportions from two independent binomial populations.
🔑 Definition — Sampling Distribution of p̂₁ - p̂₂: The probability distribution of the differences between the proportions of success from all possible pairs of independent samples drawn from two binomial populations with proportions p₁ and p₂.
Properties of the Sampling Distribution of p̂₁ - p̂₂:
Property No. 1 (Mean): μₚ̂₁₋ₚ̂₂ = p₁ - p₂
Property No. 2 (Standard Error): σₚ̂₁₋ₚ̂₂ = √(p₁q₁/n₁ + p₂q₂/n₂), where q = 1 - p
📌 Example: It is claimed that 30% of households in Community A and 20% in Community B have at least one teenager. A sample of 100 households from each community yields p̂ₐ = 0.34 and p̂_B = 0.13. What is the probability of observing a difference this large or larger if the claims are true?
Step 1: State the population parameters: pₐ = 0.30, p_B = 0.20 (claims assumed true) nₐ = n_B = 100
Step 2: Compute the mean of the sampling distribution: μₚ̂ₐ₋ₚ̂_B = 0.30 - 0.20 = 0.10
Step 3: Compute the standard error: σₚ̂ₐ₋ₚ̂_B = √((0.30)(0.70)/100 + (0.20)(0.80)/100) = √(0.21/100 + 0.16/100) = √(0.0021 + 0.0016) = √0.0037 = 0.0608
Step 4: Observed difference: p̂ₐ - p̂_B = 0.34 - 0.13 = 0.21
Step 5: Compute z-value: z = (0.21 - 0.10) / 0.0608 = 0.11 / 0.0608 = 1.83
Step 6: Find the probability: From the standard normal table, area between z = 0 and z = 1.83 is 0.4664. P(p̂ₐ - p̂_B ≥ 0.21) = P(Z ≥ 1.83) = 0.5 - 0.4664 = 0.0336
The probability is 3.36%. If the claims are true, the chance of observing a difference as large as or larger than 0.21 is only 3.36%.
Note on Sampling Distributions:
- All discussed sampling distributions assume simple random sampling
- This technique is appropriate when the population is homogeneous
Why "Standard Error"? The term "standard error" is used because any statistic considered as an estimate of a population parameter will have a difference (error) from the true parameter value. The standard deviation of the sampling distribution provides a "standard" value of this error.
Point Estimation
This section introduces the fundamental concepts of estimating population parameters using sample data.
🔑 Definition — Point Estimation: The process of providing a single value calculated from a sample that is likely to be close in magnitude to the unknown population parameter.
Difference Between 'Estimate' and 'Estimator':
- Estimator: The rule, method, or formula used to estimate a parameter (a random variable)
- Estimate: The numerical value obtained by substituting sample observations into the estimator formula
📌 Example: For a random sample X₁, X₂, ..., Xₙ from a population with mean μ:
- Estimator: X̅ = (1/n) Σᵢ₌₁ⁿ Xᵢ
- Estimate: x̅ (the numerical value of X̅)
An estimator is always a statistic which is a function of sample observations and hence is a random variable. In repeated sampling, an estimator has a probability distribution known as its sampling distribution.
Desirable Qualities of a Good Point Estimator
The lecture covers three criteria for evaluating estimators, with two discussed in detail.
1. Unbiasedness
🔑 Definition — Unbiased Estimator: An estimator θ̂ is unbiased if its expected value equals the true value of the population parameter being estimated. That is, E(θ̂) = θ.
If E(θ̂) ≠ θ, the statistic is called a biased estimator.
Examples of Unbiased Estimators:
- Sample mean X̅: E(X̅) = μ (always unbiased for μ, regardless of sample size)
- Sample median X̃: E(X̃) = μ when the population is normally distributed
- Sample proportion p̂: E(p̂) = p (unbiased estimator of population proportion)
Proof for Sample Proportion: Since X (number of successes) follows binomial distribution with E(X) = np: E(p̂) = E(X/n) = (1/n)E(X) = (1/n)(np) = p
Visual Representation: Unbiasedness means the sampling distribution of X̅ is centered at μ. Over-estimates and under-estimates balance each other out in the long run.
Sample Variance — A Special Case: The sample variance S² = (1/n) Σ(Xᵢ - X̅)² is a biased estimator of σ² because E(S²) ≠ σ².
Modified Formula for Unbiased Sample Variance: s² = Σ(xᵢ - x̅)² / (n - 1) Since E(s²) = σ², s² is an unbiased estimator of σ².
Why Unbiasedness Matters: In an actual study, we can never guarantee that our sample statistic will exactly equal the population parameter. Unbiasedness ensures that our estimation procedure (formula) is such that, in repeated sampling, the average value of our statistic equals the parameter.
Bias Direction:
- Positively biased: E(θ̂) > θ
- Negatively biased: E(θ̂) < θ
2. Consistency
🔑 Definition — Consistent Estimator: An estimator θ̂ is consistent if, for any arbitrarily small positive quantity ε: lim (n→∞) P(|θ̂ - θ| ≤ ε) = 1
In other words, the probability that θ̂ is very close to θ approaches unity as the sample size increases.
Key Properties of Consistency:
- It is a large sample property
- A consistent estimator may or may not be unbiased
Examples:
- Sample mean X̅: Consistent estimator of μ (and unbiased)
- Sample proportion p̂: Consistent estimator of p
- Sample median: NOT a consistent estimator of μ when the population has a skewed distribution
- Sample variance S²: Though biased, IS a consistent estimator of σ²
General Rule: A statistic whose standard error decreases with an increase in sample size will be consistent.
⭐ Key Takeaways
- The sampling distribution of the difference between two sample means (X̅₁ - X̅₂) is approximately normal for large samples, with mean equal to the difference in population means and standard error equal to the square root of the sum of each population variance divided by its respective sample size.
- The sampling distribution of the difference between two sample proportions (p̂₁ - p̂₂) has mean equal to p₁ - p₂ and standard error equal to √(p₁q₁/n₁ + p₂q₂/n₂), enabling probability calculations for observed differences under assumed population proportions.
- A point estimator is a random variable (a formula/rule), while an estimate is the numerical value obtained from a specific sample—this distinction is critical in statistical inference.
- Unbiasedness requires E(θ̂) = θ; the sample mean X̅, sample proportion p̂, and the modified sample variance s² (with n-1 denominator) are unbiased estimators, while S² (with n denominator) is biased.
- Consistency requires that as sample size increases, the estimator's probability of being close to the parameter approaches 1—this is a large-sample property linked to a decreasing standard error.
🧠 Quick Revision Questions
- For the battery example, what is the mean and standard error of the sampling distribution of X̅₁ - X̅₂, and what does the computed probability of 0.0329 tell us?
- What are the two key properties of the sampling distribution of p̂₁ - p̂₂, and how are they used to find the probability of observing a specific sample difference?
- What is the precise distinction between an "estimator" and an "estimate," and why is an estimator considered a random variable?
- Explain why the sample variance S² = (1/n)Σ(Xᵢ - X̅)² is a biased estimator of σ², and how the formula is modified to achieve unbiasedness.
- Define consistency and explain why the sample median is not a consistent estimator of μ when the population distribution is skewed, but the sample mean is consistent.
📘 Lecture 34 — Desirable Qualities of a Good Point Estimator: Efficiency; Methods of Point Estimation: The Method of Moments, The Method of Least Squares, The Method of Maximum Likelihood; Interval Estimation: Confidence Interval for μ
📖 Overview: This lecture completes the discussion of desirable qualities of point estimators by introducing efficiency, then presents three major methods of point estimation: the Method of Moments, Method of Least Squares, and Method of Maximum Likelihood. The lecture concludes by introducing interval estimation, developing the 95% confidence interval for the population mean μ using the normal distribution.
🗂️ Topics Covered
The lecture covers efficiency as a desirable quality of point estimators, followed by three methods of point estimation: the Method of Moments (with examples for uniform and normal distributions), the Method of Least Squares (linked to regression), and the Method of Maximum Likelihood (with examples for Poisson, geometric, Bernoulli, exponential, and normal distributions). The lecture then transitions to interval estimation, explaining the need for confidence intervals and deriving the 95% confidence interval for μ using the standard normal distribution and estimated standard deviation.
📝 Lecture Summary
Desirable Qualities of a Good Point Estimator: Efficiency
An unbiased estimator is defined to be efficient if the variance of its sampling distribution is smaller than that of the sampling distribution of any other unbiased estimator of the same parameter. In other words, suppose that there are two unbiased estimators T₁ and T₂ of the same parameter θ. Then, the estimator T₁ will be said to be more efficient than T₂ if Var(T₁) < Var(T₂).
The relative efficiency of T₁ compared to T₂ (where both are unbiased estimators) is given by the ratio: 🔑 Definition — Relative Efficiency: E = Var(T₂) / Var(T₁) If multiplied by 100, we obtain the relative efficiency in percentage form.
📌 Example: Both the sample mean and the sample median for a population that has a normal distribution are unbiased and consistent estimators of μ, but the variance of the sampling distribution of sample means is smaller than the variance of the sampling distribution of sample medians. Hence, the sample mean is more efficient than the sample median as an estimator of μ.
The Method of Moments
The Method of Moments, due to Karl Pearson (1857-1936), consists of calculating a few moments of the sample values and equating them to the corresponding moments of a population, thus getting as many equations as needed to solve for the unknown parameters.
Let X₁, X₂, ..., Xₙ be a random sample of size n from a population. Then the rth sample moment about zero is: 🔑 Definition — Sample Moment: m'ᵣ = (∑Xᵢʳ) / n, r = 1, 2, ... The corresponding rth population moment is μ'ᵣ. We match these moments and get as many equations as needed to solve for the unknown parameters.
📌 Example 1: Let X be uniformly distributed on the interval (0, θ). Find an estimator of θ by the method of moments. Solution: The probability density function is f(x) = 1/θ, 0 ≤ x ≤ θ. The first sample moment about zero is m'₁ = (∑Xᵢ)/n = X̄. The first population moment about zero is μ'₁ = ∫₀^θ x·(1/θ) dx = [x²/(2θ)]₀^θ = θ/2. Matching: X̄ = θ/2, so θ̂ = 2X̄. Hence, the moment estimator of θ is twice the sample mean.
📌 Example 2: Let X₁, X₂, ..., Xₙ be a random sample of size n from a normal population with parameters μ and σ². Find these parameters by the method of moments. Solution: The first two sample moments: m'₁ = (1/n)∑Xᵢ = X̄ and m'₂ = (1/n)∑Xᵢ². The corresponding moments of a normal distribution: μ'₁ = μ and μ'₂ = σ² + μ². Matching: μ̂ = (1/n)∑Xᵢ = X̄, and σ̂² = (∑Xᵢ²)/n - X̄² = (1/n)∑(Xᵢ - X̄)² = S².
💡 Why this matters: A shortcoming of this method is that the moment estimators are, in general, inefficient.
The Method of Least Squares
The Method of Least Squares, due to Gauss (1777-1855) and Markov (1856-1922), is based on the theory of linear estimation. An estimator found by minimizing the sum of squared deviations of the sample values from some function hypothesized as a fit for the data is called the least squares estimator.
This method was discussed in connection with regression analysis (Lecture No. 15). When fitting a straight line y = a + bx to real data, 'a' and 'b' are determined by minimizing the sum of squared deviations between the fitted line and the data-points. The y-intercept and slope 'a' and 'b' are least-square estimates of the y-intercept and slope of the true line for the entire population.
Method of Maximum Likelihood
The Method of Maximum Likelihood, regarded as the most important method of estimation, was introduced in 1922 by Sir Ronald A. Fisher (1890-1962).
Rationale: "To consider every possible value that the parameter might have, and for each value, compute the probability that the given sample would have occurred if that were the true value of the parameter. That value of the parameter for which the probability of a given sample is greatest, is chosen as an estimate." An estimate obtained by this method is called the maximum likelihood estimate (MLE). This method is applicable to both discrete and continuous random variables.
EXAMPLES OF MLE's IN CASE OF DISCRETE DISTRIBUTIONS:
- Poisson distribution (P(X=x) = e^(-μ)μ^x / x!, x=0,1,2,...): MLE of μ is X̄ (sample mean)
- Geometric distribution: MLE of p is 1/X̄ (reciprocal of the mean)
- Bernoulli distribution (P(X=x) = p^x q^(1-x), x=0,1): MLE of p is X̄ (sample mean)
EXAMPLES OF MLE's IN CASE OF CONTINUOUS DISTRIBUTIONS:
- Exponential distribution (f(x) = θe^(-θx), x>0, θ>0): MLE of θ is 1/X̄ (reciprocal of sample mean)
- Normal distribution (parameters μ and σ²): Joint ML estimators of μ and σ² are X̄ (sample mean) and S² (sample variance, which is not an unbiased estimator of σ²)
💡 Why this matters: Maximum Likelihood Estimators are consistent and efficient but not necessarily unbiased (as S² is not unbiased).
📌 Example: Human weight is approximately normally distributed. A random sample of 15 adult males yields weights (in pounds): 131.5, 136.9, 133.8, 130.1, 133.9, 135.2, 129.6, 134.4, 130.5, 134.2, 131.6, 136.7, 135.8, 134.5, 132.7. Find the MLEs for μ and σ². Solution: X̄ = 133.43 and S² = 5.10. These are the joint Maximum Likelihood Estimates of the mean and variance.
Interval Estimation: Confidence Interval for μ
Point estimation gives a single quantity computed from the sample as an estimate of a population parameter. The limitation is that we have no way of ascertaining how close our point estimate is to the true value.
Interval estimation is a procedure of estimating the unknown parameter which specifies a range of values within which the parameter is expected to lie. A confidence interval is an interval computed from the sample observations with a statement of how confident we are that the interval does contain the population parameter.
The lecture uses the Ministry of Transport test example (faults per car, uniformly distributed 0-4) to show that as sample size increases, the sampling distribution of X̄ becomes approximately normal.
For large samples from an infinite population, X̄ is normally distributed with mean μ and variance σ²/n, i.e., X̄ ~ N(μ, σ²/n). The standardized version Z = (X̄ - μ) / (σ/√n) is N(0,1).
For the standard normal distribution: 🔑 Probability Statement: P(-1.96 < Z < 1.96) = 0.95
This leads to: P(-1.96 ≤ (X̄ - μ) / (σ/√n) ≤ 1.96) = 0.95
Rearranging: P(X̄ - 1.96(σ/√n) ≤ μ ≤ X̄ + 1.96(σ/√n)) = 0.95
The 95% confidence interval for μ is: 📐 Formula: X̄ ± 1.96(σ/√n)
Since the population standard deviation σ is usually unknown, we estimate it. The quantity s² = ∑(Xᵢ - X̄)² / (n-1) is an unbiased estimator of σ².
When σ is unknown, the 95% Confidence Interval for μ is: 📐 Formula: X̄ ± 1.96(s/√n) where s/√n is the standard error of the sample mean.
The points X̄ - 1.96(s/√n) and X̄ + 1.96(s/√n) are called the lower and upper limits of the 95% confidence interval.
⭐ Key Takeaways
You must remember that efficiency compares unbiased estimators by their sampling variance — the one with smaller variance is more efficient, and the sample mean is more efficient than the sample median for normal populations. The three methods of point estimation each have distinct approaches: the Method of Moments equates sample moments to population moments (giving θ̂ = 2X̄ for uniform(0,θ) and X̄, S² for normal), the Method of Least Squares minimizes sum of squared deviations (used in regression), and the Method of Maximum Likelihood maximizes the likelihood function (giving MLEs that are consistent and efficient but not necessarily unbiased). For the normal distribution, the MLEs are X̄ and S². Finally, the 95% confidence interval for μ is X̄ ± 1.96(σ/√n) when σ is known, and X̄ ± 1.96(s/√n) when σ is estimated by s, with the interpretation that we are 95% confident the interval contains the true population mean.
🧠 Quick Revision Questions
- What does it mean for one unbiased estimator to be more efficient than another?
- For a uniform distribution on (0, θ), what is the method of moments estimator of θ, and how is it derived?
- What is the rationale behind the method of maximum likelihood?
- What are the maximum likelihood estimators of μ and σ² for a normal distribution, and are they unbiased?
- Write the formula for the 95% confidence interval for μ when the population standard deviation σ is unknown. What quantity is used to estimate σ?
📘 Lecture 35 — Confidence Interval for μ (continued). Confidence Interval for μ₁-μ₂.
📖 Overview: This lecture extends the construction of confidence intervals for a single population mean (μ) with detailed examples and important interpretations regarding the meaning of confidence levels. It then introduces the concept of confidence intervals for the difference between two population means (μ₁-μ₂), providing formulas and worked examples for comparing two distinct groups.
🗂️ Topics Covered
The lecture continues the discussion on confidence intervals for the mean of a population (μ), presenting examples with large samples to construct 95%, 90%, and 99% intervals. It covers the crucial interpretation of confidence in repeated sampling, defines a confidence interval as a point estimate plus/minus a number of standard errors, and discusses the trade-offs between confidence level and interval width. The second half introduces the formula and examples for the confidence interval for the difference between two population means (μ₁-μ₂) using large, independent samples.
📝 Lecture Summary
Confidence Interval for μ (continued).
This section provides worked examples of constructing confidence intervals for the population mean μ. It emphasizes the use of the sample standard deviation (s) as an estimate for the population standard deviation (σ) when the sample size is large (n ≥ 30). The lecture also details the correct interpretation of a 95% confidence interval, clarifying that the probability statement applies to the procedure in repeated sampling, not to a single, specific interval.
🔑 Definition — Point Estimate: The sample mean (X̄) is the best single-number estimate of the population mean μ.
📐 Formula: For large n (n ≥ 30), the confidence interval for μ is X̄ ± z_(α/2) * (s / √n), where z_(α/2) is the critical value from the standard normal distribution.
📌 Example (95% C.I.): A sample of 500 employees has X̄ = 5.38 days and s = 3.53 days. The 95% confidence interval is 5.38 ± 1.96*(3.53/√500) = 5.38 ± 0.31, resulting in an interval of (5.07, 5.69) days.
💡 Why this matters: The statement "the mean lies between 5.07 and 5.69 days with 95% confidence" means that if we were to take many such samples and compute intervals for each, approximately 95% of those intervals would contain the true population mean μ.
📌 Example (90% C.I.): A sample of 64 days on a bridge has X̄ = 5410 and s = 680. The 90% confidence interval is 5410 ± 1.645*(680/√64) = 5410 ± 139.8, resulting in an interval of (5270.2, 5549.8) or (5270, 5550).
📌 Example (99% C.I.): A sample of 110 rental cars has X̄ = 85.5 miles and s = 19.3 miles. The 99% confidence interval is 85.5 ± 2.575*(19.3/√110) = 85.5 ± 4.7, resulting in an interval of (80.8, 90.2) miles per day.
An Important Way of Interpreting a Confidence Interval
This section reframes the confidence interval formula as a point estimate plus/minus a multiple of the standard error of the estimate. It explains that the "multiple" (z_(α/2)) determines the level of confidence, and that a narrower interval with high confidence is preferable. The lecture discusses the two key factors that affect the width of the interval: the sample size (n) and the level of confidence (1-α).
📐 Formula (Reframed): The confidence interval for μ can be defined as X̄ ± z_(α/2) * σ_X̄, where σ_X̄ (sigma sub X-bar) is the standard error of the mean, equal to σ/√n.
📌 Key Concept: A narrow confidence interval is better. It is achieved by making z_(α/2) * σ_X̄ small. This happens when n is large (making σ_X̄ small) or when the confidence level 1-α is low (making z_(α/2) small). In practice, a compromise must be struck between the desired level of confidence and the acceptable width of the interval.
Confidence Interval for the Difference Between the Means of Two Populations
This section introduces the formula for constructing a confidence interval for the difference between two population means (μ₁-μ₂) using large samples drawn independently from two populations.
🔑 Definition — Difference of Means (μ₁-μ₂): The parameter of interest is the real difference between the average values of two distinct populations.
📐 Formula: For large, independent samples, the confidence interval for μ₁ - μ₂ is (X̄₁ - X̄₂) ± z_(α/2) * √( (s₁² / n₁) + (s₂² / n₂) ).
📌 Example 1 (90% C.I. for Income Difference): Factory A (n=160, X̄=12.80, s²=64) and Factory B (n=220, X̄=11.25, s²=47). The 90% C.I. is (12.80 - 11.25) ± 1.645 * √(64/160 + 47/220) = 1.55 ± 1.28, resulting in an interval of (0.27, 2.83). We are 90% confident the true difference in weekly incomes is between Rs. 0.27 and Rs. 2.83.
📌 Example 2 (98% C.I. for Coupon Savings Difference): Middle-income shoppers (n₁=60, X̄₁=$5.84, s₁=$1.41) and Low-income shoppers (n₂=80, X̄₂=$2.67, s₂=$0.54). The 98% C.I. is (5.84 - 2.67) ± 2.33 * √((1.41²/60) + (0.54²/80)) = 3.17 ± 0.45, resulting in an interval of (2.72, 3.62). The data strongly indicates a real difference as the interval does not contain zero.
⭐ Key Takeaways
A confidence interval for a mean (μ) has the form point estimate ± margin of error, where the margin of error is the critical value (z) times the standard error. A 95% confidence interval means that in repeated sampling, 95% of such intervals will contain the true population mean; it does not mean there is a 95% probability for any single computed interval. The width of a confidence interval is inversely related to sample size and directly related to the desired level of confidence. For comparing two population means (μ₁-μ₂) with large, independent samples, the same logic applies, and the interval formula uses the difference of sample means and the combined standard error. If the confidence interval for a difference does not include zero, it provides evidence that a real difference between the two population means exists.
🧠 Quick Revision Questions
- What is the formula for a (1-α)100% confidence interval for a population mean (μ) when the sample size is large (n ≥ 30)?
- Explain the correct interpretation of a "95% confidence interval" for a population mean.
- List the two primary factors that affect the width of a confidence interval and state how changing each factor affects the width.
- What is the formula for constructing a confidence interval for the difference between two population means (μ₁-μ₂) when using large, independent samples?
- In Example 2 for the difference of means (coupon savings), the 98% confidence interval was ($2.72, $3.62). What can you conclude about the difference between the means of the two populations?
📘 Lecture 36 — Large Sample Confidence Intervals for p and p1-p2, Determination of Sample Size (with reference to Interval Estimation), Hypothesis-Testing (An Introduction)
📖 Overview: This lecture extends the concept of confidence intervals to population proportions and the difference between two proportions. It then addresses the critical practical question of determining the required sample size for estimation. Finally, it introduces the fundamental concepts of hypothesis-testing, which, alongside estimation, forms the core of statistical inference.
🗂️ Topics Covered
The lecture begins with constructing and interpreting large-sample confidence intervals for a single population proportion (p) and for the difference between two population proportions (p1-p2), illustrated with detailed examples. It then explains how to calculate the necessary sample size for estimating a population mean and a population proportion with a desired precision and confidence level. The lecture concludes by defining what a statistical hypothesis is and introducing the concepts of null and alternative hypotheses, Type-I and Type-II errors, and the level of significance.
📝 Lecture Summary
Large Sample Confidence Intervals for p and p1-p2
The first major topic is constructing confidence intervals for a population proportion, p, which represents the proportion of successes in a binomial population. For a large sample, the confidence interval for p is given by p̂ ± z(α/2) √[p̂(1-p̂)/n], where p̂ is the sample proportion of successes, n is the sample size, and z(α/2) is the critical value from the standard normal distribution (1.96 for 95% confidence, 2.58 for 99% confidence). A sample is considered "large enough" if both np̂ and n(1-p̂) are greater than or equal to 5.
🔑 Definition — Large Sample Confidence Interval for p: A range of values, calculated from sample data, that is likely to contain the true population proportion p with a specified level of confidence.
📐 Formula: p̂ ± z(α/2) * √[p̂(1-p̂)/n]
→ This formula calculates the margin of error around the point estimate (p̂) to create an interval that is expected to contain the true population proportion p a certain percentage of the time.
📌 Example (EXAMPLE-1):
- Context: A survey of 634 teenagers with three or more juvenile court appearances found 291 were orphans.
- Goal: Estimate the proportion of all such teenagers who are orphans with 99% confidence.
- Steps:
- Identify variables: n = 634, number of successes = 291.
- Calculate sample proportion (p̂): p̂ = 291/634 = 0.459. Then q̂ = 1 - 0.459 = 0.541.
- Find critical value: For 99% confidence, z(α/2) = 2.58.
- Calculate standard error: √[p̂*q̂/n] = √[(0.459 * 0.541) / 634].
- Compute margin of error: 2.58 * standard error = 0.051.
- Construct interval: 0.459 ± 0.051 = (0.408, 0.510).
- Conclusion: We are 99% confident that the true proportion of these teenagers who are orphans is between 40.8% and 51.0%.
📌 Example (EXAMPLE-2):
- Context: A poll of 500 voters found 290 favor Mr. Scott.
- Goal: Develop a 95% confidence interval for the proportion of all voters who will vote for Mr. Scott and determine if he will win.
- Steps:
- Identify variables: n = 500, number of successes = 290. p̂ = 290/500 = 0.58.
- Find critical value: For 95% confidence, z(α/2) = 1.96.
- Calculate standard error: √[0.58*(1-0.58)/500] = √[(0.58*0.42)/500].
- Compute margin of error: 1.96 * standard error = 0.043.
- Construct interval: 0.58 ± 0.043 = (0.537, 0.623).
- Conclusion: Since the entire confidence interval (0.537 to 0.623) is above 0.50, we conclude that Mr. Scott will likely win the election.
📌 Example (EXAMPLE-3):
- Context: A survey of 210 chief executives found that 51% had a management-succession plan.
- Goal: Compute a 92% confidence interval to estimate the true proportion of all fast-growing small companies that have such a plan.
- Steps:
- Identify variables: n = 210, p̂ = 0.51, q̂ = 0.49.
- Find critical value: For 92% confidence, z(α/2) = 1.75.
- Calculate standard error: √[(0.51*0.49)/210].
- Compute margin of error: 1.75 * standard error = 0.06.
- Construct interval: 0.51 ± 0.06 = (0.45, 0.57).
- Conclusion: We are 92% confident that the true proportion is between 0.45 and 0.57.
The lecture then shifts to the confidence interval for the difference in two population proportions, (p1-p2). For large independent samples from two binomial populations, the confidence interval is (p̂1 – p̂2) ± z(α/2) √[ p̂1(1-p̂1)/n1 + p̂2(1-p̂2)/n2 ].
📐 Formula: (p̂1 – p̂2) ± z(α/2) * √[ p̂1(1-p̂1)/n1 + p̂2(1-p̂2)/n2 ]
→ This formula estimates the difference between two population proportions by creating an interval around the observed difference in sample proportions.
📌 Example:
- Context: Among college students, 300 of 400 resident students and 200 of 300 non-resident students approved a course of action.
- Goal: Estimate the difference in proportions (resident minus non-resident) and compute a 90% confidence interval.
- Steps:
- Identify variables: n1=400, successes1=300, p̂1=0.75; n2=300, successes2=200, p̂2=0.67.
- Find critical value: For 90% confidence, z(α/2) = 1.645.
- Calculate standard error: √[(0.750.25)/400 + (0.670.33)/300] = √[0.00046875 + 0.000737] = √0.00120575 ≈ 0.0347.
- Compute margin of error: 1.645 * 0.0347 = 0.057.
- Construct interval: (0.75 – 0.67) ± 0.057 = 0.08 ± 0.057 = (0.023, 0.137).
- Conclusion: We are 90% confident that the true difference in proportions is between 2.3% and 13.7%.
Determination of Sample Size (with reference to Interval Estimation)
This section addresses the critical question: "How large should the sample be?" The goal is to find a sample size that provides a desired level of precision (error bound, e) with a desired level of confidence.
For estimating a population mean (μ), the required sample size is derived from the error bound formula for a confidence interval. The maximum allowable error, e, is given by e = z(α/2) * σ/√n. Solving for n yields the formula: n = (z(α/2) * σ / e)². Since the population standard deviation σ is usually unknown, an estimate is used, often from a pilot study or past data. The computed value for n is always rounded up to the next integer.
📐 Formula: n = (z(α/2) * σ / e)²
→ This formula calculates the minimum sample size needed to estimate a population mean with a specified margin of error (e) and confidence level (determined by z(α/2)).
📌 Example (for population mean):
- Context: Estimate the mean of a population with 95% confidence so that sample mean differs from the true mean by no more than 25% of the standard deviation.
- Steps:
- Identify variables: e = (25%) of σ = σ/4, z(α/2) for 95% confidence = 1.96.
- Apply formula: n = (1.96 * σ / (σ/4))² = (1.96 * 4)² = 61.4656.
- Conclusion: The required sample size is the next highest integer, 62.
For estimating a population proportion (p), the error bound is e = z(α/2) * √[p̂*q̂/n] . Solving for n yields: n = [(z(α/2))² * p̂ * q̂] / e². The values for p̂ and q̂ are unknown before sampling, so a pilot sample estimate is used. If no estimate is available, the most conservative value, p̂=0.5, is used as it maximizes the product p̂*q̂.
📐 Formula: n = [(z(α/2))² * p̂ * q̂] / e²
→ This formula calculates the minimum sample size needed to estimate a population proportion with a specified margin of error and confidence level.
📌 Example (for population proportion):
- Context: In a pilot sample of 75 axle shafts, 12 had a rough surface. We want to be 95% confident that the error in estimating p is less than 0.05.
- Steps:
- Identify variables: p̂ = 12/75 = 0.16, q̂ = 1-0.16 = 0.84, e = 0.05, z(α/2) for 95% = 1.96.
- Apply formula: n = (1.96)² / (0.05)² * (0.16 * 0.84) = (3.8416 / 0.0025) * 0.1344 = 1536.64 * 0.1344 = 206.52.
- Conclusion: The required sample size is 207.
Hypothesis-Testing (An Introduction)
This section introduces hypothesis-testing, the second major branch of statistical inference. A statistical hypothesis is a statement or assumption about a population parameter that may or may not be true. The goal of hypothesis testing is to decide, based on sample data, whether to accept or reject such a hypothesis. It is crucial to understand that "accepting" a hypothesis means there is insufficient evidence to reject it, not that it is proven true. "Rejecting" a hypothesis means declaring it false.
🔑 Definition — Null Hypothesis (H0): The hypothesis to be tested for possible rejection or nullification. It is a precise statement, often about no change, no effect, or a specific numerical value. For example, H0: μ = 62 inches.
🔑 Definition — Alternative Hypothesis (H1 or HA): The hypothesis that is accepted when the null hypothesis is rejected. It represents the research claim or suspected difference. For example, H1: μ ≠ 62 inches or H1: μ < 62 inches.
💡 Why this matters: In any hypothesis test, H0 and H1 are mutually exclusive and exhaustive statements about the population parameter. The sample data is used to decide which one is more plausible.
The decision to reject or accept H0 involves a risk of error.
🔑 Definition — Type-I Error: The error of rejecting a null hypothesis when it is actually true. It is also called an error of the first kind.
🔑 Definition — Type-II Error: The error of accepting (not rejecting) a null hypothesis when it is actually false. It is also called an error of the second kind.
🔑 Definition — Level of Significance (α): The probability of committing a Type-I error, P(reject H0 | H0 is true). The probability of a Type-II error is denoted by β.
| Decision | H0 is True (Reality) | H0 is False (Reality) |
|---|---|---|
| Accept H0 | Correct Decision (No Error) | Wrong Decision (Type-II Error) |
| Reject H0 | Wrong Decision (Type-I Error) | Correct Decision (No Error) |
📌 Example: A judge presumes a defendant is innocent (H0: Innocent).
- Type-I Error: The jury rejects H0 and convicts an innocent person.
- Type-II Error: The jury accepts H0 and acquits a guilty person.
⭐ Key Takeaways
The key learning from this lecture is the ability to construct confidence intervals for proportions and understand the planning needed before data collection. You must be able to apply the correct formula for a single proportion p or the difference between two proportions (p1-p2), and correctly interpret the resulting interval. Equally critical is understanding how sample size determination works, using the formulas n = (zσ/e)² for a mean and n = (z² p̂ q̂)/e² for a proportion, and always rounding up. Finally, you should be able to define the null and alternative hypotheses, and distinguish between Type-I (rejecting a true H0) and Type-II (failing to reject a false H0) errors, as these concepts form the foundation for all hypothesis testing.
🧠 Quick Revision Questions
- What are the two conditions that must be met for a sample to be considered "large enough" to construct a confidence interval for a population proportion (p)?
- A researcher wants to estimate the difference in approval rates for a policy between two cities, with a 90% confidence. What is the value of z(α/2) they should use?
- Explain the difference between a Type-I error and a Type-II error in the context of testing a new drug where H0 is that the drug is ineffective.
- A researcher needs to estimate a population mean with a margin of error of 2 units and is 99% confident. If a known standard deviation is 8, what sample size is required?
- You want to estimate the proportion of students who own a car with a 95% confidence and a 3% margin of error. If no pilot study is available, what conservative value of p̂ should you use, and why?
📘 Lecture 37 — Hypothesis-Testing (continuation of basic concepts) & Hypothesis-Testing regarding μ (based on Z-statistic)
📖 Overview: This lecture continues the discussion of hypothesis-testing by introducing the test-statistic, acceptance and rejection regions, and the critical region. It explains the crucial distinction between one-tailed and two-tailed tests and provides detailed, step-by-step examples of hypothesis testing for the population mean using the Z-statistic.
🗂️ Topics Covered
This lecture covers the concept of a test-statistic and its sampling distribution, the division of test-statistic values into acceptance and rejection regions defined by critical values, the relationship between the level of significance (α) and the critical region, and the detailed methodology of one-tailed and two-tailed tests. It concludes with two comprehensive solved examples: a two-tailed test regarding a change in weekly desk production and a one-tailed test regarding day-care costs.
📝 Lecture Summary
Basic Concepts of Hypothesis-Testing (Continuation)
TEST-STATISTIC A test-statistic is a statistic (a function of the sample data not containing any parameters) that provides a basis for testing a null hypothesis. Every test-statistic has a probability distribution (its sampling distribution) which gives the probability that the test-statistic will assume a value greater than or equal to a specified value (or less than or equal to a specified value) when the null hypothesis is true.
ACCEPTANCE AND REJECTION REGIONS All possible values a test-statistic can assume are divided into two mutually exclusive groups. One group consists of values consistent with the null hypothesis, called the acceptance region. The other group leads to the rejection of the null hypothesis and is called the rejection region or critical region. The value(s) separating the critical region from the acceptance region are called the critical value(s). The critical value is decided by the experimenter, with the most frequently used significance levels (α) being 0.05 (5%) and 0.01 (1%). A significance level of α = 5% means there are about 5 chances in 100 of incorrectly rejecting a true null hypothesis (a Type-I error).
RELATIONSHIP BETWEEN THE LEVEL OF SIGNIFICANCE AND THE CRITICAL REGION The level of significance (α) is the basis for determining the critical region of the test. For example, testing H0: μ = 45 against H1: μ ≠ 45 with a 5% significance level and a standard normal Z test-statistic, the critical values are Z = ±1.96. This means the total area of the rejection region (α) is 0.05, equally divided as 0.025 in each tail of the distribution. 💡 Why this matters: The significance level is a direct input for finding the critical values, which define the decision rule for the hypothesis test.
- 🔑 Definition — Test-Statistic: A function of the sample data used to decide whether to reject the null hypothesis.
- 🔑 Definition — Critical Region (Rejection Region): The set of values for the test-statistic that leads to the rejection of H0.
- 🔑 Definition — Critical Value(s): The value(s) that separate the critical region from the acceptance region.
- 📐 Formula — Critical Value for Two-Tailed Test (α=0.05): Z = ±1.96 → The test-statistic must be less than -1.96 or greater than +1.96 to reject H0.
ONE-TAILED AND TWO-TAILED TESTS
One-Tailed (or One-Sided) Test A test where the entire rejection region lies in only one tail of the sampling distribution. This is used when the alternative hypothesis (H1) states a direction: H1: θ > θ0 (right-tailed) or H1: θ < θ0 (left-tailed). For example, for a population mean with a large sample, H1: μ > μ0 uses a right-tailed test.
- For H1: μ < μ0: Reject H0 if the computed Z < –zα.
- For H1: μ > μ0: Reject H0 if the computed Z > zα.
Two-Tailed (or Two-Sided) Test A test where the rejection region is divided equally between the two tails of the sampling distribution. This is used when the alternative hypothesis does not state a direction, e.g., H1: μ ≠ μ0 (meaning H1: μ < μ0 or μ > μ0).
- For H1: μ ≠ μ0: Reject H0 if the computed Z < –zα/2 or if the computed Z > zα/2.
The location of the critical region is determined only after H1 has been stated. One-tailed and two-tailed tests differ only in the location of the critical region, not in its total size (α).
EXAMPLE (Two-Tailed Test)
A steel company’s weekly production of Model A desks has a historical mean of μ=200 and standard deviation σ=16. The vice president wants to know if the mean production last year (n=50, X̄=203.5) differs from 200 at the 0.05 significance level.
SOLUTION:
Step-1: Formulation of Hypotheses H0: μ = 200 (The population mean is 200) H1: μ ≠ 200 (The mean is different from 200) (Note: This is a two-tailed test as no direction is stated.)
Step-2: Level of Significance α = 0.05
Step-3: Test Statistic For a large sample mean: Z = (X̄ – μ) / (σ / √n)
Step-4: Calculations n = 50, X̄ = 203.5, σ = 16 Computed Z = (203.5 – 200) / (16 / √50) = 3.5 / (16 / 7.071) = 3.5 / 2.263 = 1.55
Step-5: Critical Region For a two-tailed test at α=0.05, the critical values are +1.96 and -1.96. The decision rule is: Reject H0 if the computed Z is not between -1.96 and +1.96.
Step-6: Conclusion The computed Z (1.55) lies between -1.96 and +1.96, so H0 is not rejected. The sample evidence does not show that the production rate has changed from 200 per week. The difference of 3.5 units is attributed to chance.
- 📌 Example: The computed z-value of 1.55 was less than the critical value of 1.96, leading to a failure to reject the null hypothesis.
EXAMPLE (One-Tailed Test)
A random sample of 100 workers shows a mean day-care cost of Rs.2650 with a standard deviation of Rs.500. Verify the department’s claim that the mean exceeds Rs.2500 at the 0.05 level.
SOLUTION:
Step-1: Formulation of Hypotheses The claim (mean exceeds Rs.2500) is H1. The negation (mean ≤ Rs.2500) is H0, which contains the equal sign. H0: μ ≤ 2500 H1: μ > 2500 (exceeds 2500)
Step-2: Level of Significance α = 0.05
Step-3: Test Statistic The test-statistic, under H0, is Z = (X̄ – μ0) / (S / √n), which is approximately normal for a large sample (n=100).
Step-4: Critical Region For this right-tailed test, the rejection region is Z > Z0.05 = 1.645.
Step-5: Calculations n = 100, X̄ = 2650, S = 500, μ0 = 2500 Computed Z = (2650 – 2500) / (500 / √100) = 150 / (500 / 10) = 150 / 50 = 3
Step-6: Conclusion The computed Z (3) is greater than the critical value (1.645), so it falls in the rejection region. Therefore, H0 is rejected, and the department’s claim is supported by the sample evidence.
An Interesting and Important Point: For α = 0.01, Zα = 2.33. Since the computed Z (3) is even greater than 2.33, the computed value of X̄ is highly significant. Even with only a 1% chance of being wrong, the department’s claim was correct.
- 📌 Example: The computed z-value of 3 was greater than the critical value of 1.645, leading to the rejection of the null hypothesis.
⭐ Key Takeaways
The most critical points from this lecture are: 1) The test-statistic and its sampling distribution are the mathematical basis for hypothesis testing. 2) The level of significance (α) directly determines the critical value(s) which separate the acceptance and rejection regions. 3) The form of the alternative hypothesis (H1) dictates whether a one-tailed or two-tailed test is used, which changes the location of the critical region. 4) The six-step procedure for hypothesis testing provides a structured framework for making data-driven decisions. 5) A computed test-statistic value can be significant at one level (α=0.05) and highly significant at a stricter level (α=0.01), providing stronger evidence against the null hypothesis.
🧠 Quick Revision Questions
- What are the six steps in the formal procedure for hypothesis testing presented in this lecture?
- For a two-tailed test at the 0.01 significance level, what would be the critical Z-values for the test-statistic?
- How does the location of the critical (rejection) region differ between a one-tailed test (where H1: μ > μ0) and a two-tailed test (where H1: μ ≠ μ0)?
- In the first example, the computed Z-statistic was 1.55. Was the null hypothesis rejected or not, and what was the conclusion regarding the change in weekly production?
- In the second example explained why the computed Z-statistic of 3 is considered "highly significant" at the 1% level?