π Lecture 17 β COUNTING RULES
π Overview: This lecture covers the fundamental counting rules used to calculate probabilities, specifically permutations and combinations. It then introduces the foundational concepts of probability theory, including random experiments, sample spaces, and types of events. Understanding these concepts is crucial for correctly computing probabilities in a wide range of statistical problems.
ποΈ Topics Covered
The lecture reviews counting rules (multiple choice, permutations, combinations), then defines factorial notation. It explains the rule of permutations for distinct and non-distinct objects, followed by the rule of combinations and its properties as binomial coefficients. The lecture then transitions to probability concepts, defining random experiments, sample spaces, events (simple, compound, complementary), and the key event relationships: mutually exclusive, exhaustive, and equally likely events.
π Lecture Summary
COUNTING RULES
As discussed in the last lecture, there are certain rules that facilitate the calculations of probabilities in certain situations. They are known as counting rules and include concepts of; Multiple Choice, Permutations, and Combinations. The rule of multiplication was already discussed in the last lecture. Let us now consider the rule of permutations.
RULE OF PERMUTATION
A permutation is any ordered subset from a set of n distinct objects. The number of permutations of r objects, selected in a definite order from n distinct objects is denoted by the symbol ( _nP_r ) and is given by: [ _nP_r = n(n-1)(n-2)...(n-r+1) = \frac{n!}{(n-r)!} ]
FACTORIALS A factorial is the product of all positive integers less than or equal to a given positive integer. For example, ( 7! = 7 \times 6 \times 5 \times 4 \times 3 \times 2 \times 1 ). Also, we define ( 0! = 1 ).
π Formula: ( _nP_r = \frac{n!}{(n-r)!} ) β The number of ways to arrange 'r' items selected from a set of 'n' distinct items, where order matters.
π Example: A club consists of four members. How many ways are there of selecting three officers: president, secretary and treasurer? It is evident that the order, in which 3 officers are to be chosen, is of significance. Thus there are 4 choices for the first office, 3 choices for the second office, and 2 choices for the third office. Hence the total number of ways is ( 4 \times 3 \times 2 = 24 ). The same result is obtained by applying the rule of permutations: [ _4P_3 = \frac{4!}{(4-3)!} = \frac{4 \times 3 \times 2 \times 1}{1!} = 24 ] Let the four members be, A, B, C and D. A tree diagram provides an organized way of listing the possible arrangements.
In the formula of ( _nP_r ), if we put r = n, we obtain: ( _nP_n = n! ). I.e. the total number of permutations of n distinct objects, taking all n at a time, is equal to ( n! ).
π Example: Suppose that there are three persons A, B & D, and that they wish to have a photograph taken. The total number of ways in which they can be seated on three chairs is ( _3P_3 = 3! = 6 ). These are: ABD, ADB, BAD, BDA, DAB, DBA.
The above discussion pertained to the case when all the objects under consideration are distinct objects. If some of the objects are not distinct, the formula of permutations modifies as given below: The number of permutations of n objects, selected all at a time, when n objects consist of ( n_1 ) of one kind, ( n_2 ) of a second kind, ..., ( n_k ) of a kth kind, is: [ P = \frac{n!}{n_1! n_2! ... n_k!} \quad (\text{where } \sum n_i = n) ]
π Formula: ( P = \frac{n!}{n_1! n_2! ... n_k!} ) β The number of distinct arrangements of 'n' total objects where some objects are identical to each other.
π Example: How many different (meaningless) words can be formed from the word βcommitteeβ? In this example: n = 9 (total letters), ( n_1 = 1 ) (c), ( n_2 = 1 ) (o), ( n_3 = 2 ) (mβs), ( n_4 = 1 ) (i), ( n_5 = 2 ) (tβs), and ( n_6 = 2 ) (eβs). Hence, the total number of (meaningless) words is: [ P = \frac{9!}{1! 1! 2! 1! 2! 2!} = \frac{9 \times 8 \times 7 \times 6 \times 5 \times 4 \times 3 \times 2 \times 1}{1 \times 1 \times 2 \times 1 \times 1 \times 2 \times 1 \times 2 \times 1} = 45360 ]
RULE OF COMBINATION
A combination is any subset of r objects, selected without regard to their order, from a set of n distinct objects. The total number of such combinations is denoted by the symbol ( _nC_r ) or ( \binom{n}{r} ) and is given by: [ \binom{n}{r} = \frac{n!}{r! (n-r)!} ] It should be noted that ( _nP_r = r! \binom{n}{r} ). In other words, every combination of r objects (out of n objects) generates ( r! ) permutations.
π Definition β Combination: A selection of r objects from a set of n distinct objects where the order of selection does not matter.
π Formula: ( \binom{n}{r} = \frac{n!}{r!(n-r)!} ) β The number of ways to choose 'r' items from a set of 'n' distinct items, where order does not matter.
π Example: Suppose we have a group of three persons, A, B, & C. If we wish to select a group of two persons out of these three, the three possible groups are {A, B}, {A, C} and {B, C}. In other words, the total number of combinations of size two out of this set of size three is 3. Now, suppose that our interest lies in forming a committee of two persons, one of whom is to be the president and the other the secretary. The six possible committees (permutations) are: (A, B), (B, A), (A, C), (C, A), (B, C) & (C, B). The point to note is that each of the three combinations generates 2 = 2! permutations.
The quantity ( \binom{n}{r} ) or ( nC_r ) is also called a binomial coefficient because of its appearance in the binomial expansion of: [ (a+b)^n = \sum{r=0}^n \binom{n}{r} a^{n-r} b^r ] The binomial coefficient has two important properties: (i) ( \binom{n}{r} = \binom{n}{n-r} ), and (ii) ( \binom{n}{n-r} + \binom{n}{r} = \binom{n+1}{r} ) Also, it should be noted that ( \binom{n}{0} = 1 = \binom{n}{n} ) and ( \binom{n}{1} = n = \binom{n}{n-1} ).
π Example: A three-person committee is to be formed out of a group of ten persons. In how many ways can this be done? Since the order is unimportant, it is a problem involving combinations. Thus the desired number is: [ \binom{10}{3} = \frac{10!}{3! (10-3)!} = \frac{10!}{3! 7!} = \frac{10 \times 9 \times 8 \times 7!}{3 \times 2 \times 1 \times 7!} = 120 ]
π Example: In how many ways can a person draw a hand of 5 cards from a well-shuffled ordinary deck of 52 cards? The total number of ways is given by: [ \binom{52}{5} = \frac{52 \times 51 \times 50 \times 49 \times 48}{5 \times 4 \times 3 \times 2 \times 1} = 2,598,960 ]
RANDOM EXPERIMENT
Having reviewed the counting rules, let us now begin the discussion of concepts that lead to the formal definitions of probability. The term experiment means a planned activity or process whose results yield a set of data. A single performance of an experiment is called a trial. The result obtained from an experiment or a trial is called an outcome.
π Definition β Random Experiment: An experiment which produces different results even though it is repeated a large number of times under essentially similar conditions is called a Random Experiment. The tossing of a fair coin, the throwing of a balanced die, drawing of a card from a well-shuffled deck of 52 playing cards, selecting a sample, etc. are examples of random experiments.
PROPERTIES OF A RANDOM EXPERIMENT A random experiment has three properties:
- The experiment can be repeated, practically or theoretically, any number of times.
- The experiment always has two or more possible outcomes.
- The outcome of each repetition is unpredictable, i.e. it has some degree of uncertainty.
π Example: Interviewing a person to find out whether or not he or she is a smoker is an example of a random experiment. This process can be repeated a large number of times, there are at least two possible replies (βI am a smokerβ and βI am not a smokerβ), and the answer is not known in advance.
SAMPLE SPACE
π Definition β Sample Space: A set consisting of all possible outcomes that can result from a random experiment (real or conceptual) is called the sample space for the experiment and is denoted by the letter S. Each possible outcome is a member of the sample space and is called a sample point in that space.
π Example-1: The experiment of tossing a coin results in either of the two possible outcomes: a head (H) or a tail (T). The sample space is S = {H, T}.
π Example-2: The sample space for tossing two coins once (or tossing a coin twice) will contain four possible outcomes: S = {HH, HT, TH, TT}. S is the Cartesian product A Γ A, where A = {H, T}.
π Example-3: The sample space S for the random experiment of throwing two six-sided dice can be described by the Cartesian product A Γ A, where A = {1, 2, 3, 4, 5, 6}. S contains 36 outcomes or sample points.
EVENTS
π Definition β Event: Any subset of a sample space S of a random experiment is called an event. An event is an individual outcome or any number of outcomes (sample points) of a random experiment.
SIMPLE & COMPOUND EVENTS An event that contains exactly one sample point is defined as a simple event. A compound event contains more than one sample point and is produced by the union of simple events.
π Example: The occurrence of a 6 when a die is thrown is a simple event, while the occurrence of a sum of 10 with a pair of dice is a compound event, as it can be decomposed into three simple events (4, 6), (5, 5) and (6, 4).
OCCURRENCE OF AN EVENT An event A is said to occur if and only if the outcome of the experiment corresponds to some element of A.
π Example: Suppose we toss a die, and we are interested in the occurrence of an even number. If ANY of the three numbers β2β, β4β or β6β occurs, we say that the event of our interest has occurred. The event A is represented by the set {2, 4, 6}.
COMPLEMENTARY EVENT The event βnot-Aβ is denoted by ( \bar{A} ) or ( A^c ) and called the negation (or complementary event) of A.
π Example: If we toss a coin once, then the complement of βheadsβ is βtailsβ. If we toss a coin four times, then the complement of βat least one headβ is βno headsβ.
A sample space consisting of n sample points can produce ( 2^n ) different subsets (or simple and compound events). The subset that is the sample space itself is called the certain or sure event. The empty set ( \phi ) is called the impossible event.
MUTUALLY EXCLUSIVE EVENTS
π Definition β Mutually Exclusive Events: Two events A and B of a single experiment are said to be mutually exclusive or disjoint if and only if they cannot both occur at the same time, i.e. they have no points in common.
π Example-1: When we toss a coin, we get either a head or a tail, but not both at the same time. The two events 'head' and 'tail' are therefore mutually exclusive.
π Example-2: When a die is rolled, the events βeven numberβ and βodd numberβ are mutually exclusive. Three or more events originating from the same experiment are mutually exclusive if pairwise they are mutually exclusive. If the two events can occur at the same time, they are not mutually exclusive. For example, if we draw a card from an ordinary deck of 52 playing cards, it can be both a king and a diamond. Therefore, kings and diamonds are not mutually exclusive.
EXHAUSTIVE EVENTS
π Definition β Exhaustive Events: Events are said to be collectively exhaustive when the union of mutually exclusive events is equal to the entire sample space S.
π Examples: In the coin-tossing experiment, βheadβ and βtailβ are collectively exhaustive events. In the die-tossing experiment, βeven numberβ and βodd numberβ are collectively exhaustive events.
PARTITION OF THE SAMPLE SPACE A group of mutually exclusive and exhaustive events belonging to a sample space is called a partition of the sample space. With reference to any sample space S, events A and ( \bar{A} ) form a partition as they are mutually exclusive and their union is the entire sample space.
EQUALLY LIKELY EVENTS
π Definition β Equally Likely Events: Two events A and B are said to be equally likely when one event is as likely to occur as the other. In other words, each event should occur in equal number in repeated trials.
π Example: When a fair coin is tossed, the head is as likely to appear as the tail, and the proportion of times each side is expected to appear is 1/2.
π Example: If a card is drawn out of a deck of well-shuffled cards, each card is equally likely to be drawn, and the probability that any card will be drawn is 1/52.
π‘ Why this matters: The concepts of mutually exclusive, exhaustive, and equally likely events are the fundamental building blocks for the classical definition of probability, which is the ratio of favorable outcomes to the total number of equally likely outcomes. A proper understanding of these event types is essential for correctly setting up and solving probability problems.
β Key Takeaways
The critical concepts from this lecture are the distinction between permutations (ordered arrangements) and combinations (unordered selections), and the formulas to calculate their respective numbers. You must also be able to define a random experiment, its sample space, and the different types of events, including simple, compound, mutually exclusive, exhaustive, and equally likely events. A firm grasp of these definitions and formulas is essential for understanding how probabilities are formally defined and calculated in subsequent lectures.
π§ Quick Revision Questions
- What is the formula for the number of permutations of r objects chosen from n distinct objects, and how does it differ from the formula for combinations?
- How does the permutation formula change when some of the objects are not distinct?
- Define a random experiment and state its three essential properties.
- What is the difference between a simple event and a compound event? Give an example of each from the experiment of rolling a pair of dice.
- Explain the relationship between mutually exclusive and exhaustive events. What is a partition of a sample space?
π Lecture 18 β Definitions of Probability
π Overview: This lecture introduces the fundamental definitions of probability, beginning with essential concepts like mutually exclusive, exhaustive, and equally likely events. It then explores both subjective and objective approaches to probability, with a detailed focus on the classical (a priori) definition and the relative frequency definition, highlighting their applications and limitations.
ποΈ Topics Covered
The lecture begins with a review of mutually exclusive events, exhaustive events, equally likely events, and the partition of the sample space. It then introduces the subjective or personalistic approach to probability. The objective approach is divided into two parts: the classical definition of probability, with several examples and its shortcomings, followed by the relative frequency definition of probability.
π Lecture Summary
Mutually Exclusive Events
Two events A and B of a single experiment are said to be mutually exclusive or disjoint if they cannot both occur at the same time, meaning they have no points in common. For example, when tossing a coin, getting a head and getting a tail are mutually exclusive events. When rolling a die, the events βeven numberβ and βodd numberβ are mutually exclusive. Three or more events are mutually exclusive if they are pairwise mutually exclusive. If two events can occur at the same time, they are not mutually exclusive; for instance, drawing a card from a deck could result in both a king and a diamond.
π Definition β Mutually Exclusive Events: Events that cannot occur at the same time; they have no sample points in common.
π Example: In a single coin toss, the events "Head" and "Tail" are mutually exclusive because you cannot get both simultaneously.
Exhaustive Events
Events are said to be collectively exhaustive when the union of mutually exclusive events is equal to the entire sample space S. For instance, in the coin-tossing experiment, βheadβ and βtailβ are collectively exhaustive events. In the die-tossing experiment, βeven numberβ and βodd numberβ are collectively exhaustive events.
π Definition β Exhaustive Events: A set of events whose union equals the entire sample space.
Partition of the Sample Space
A group of mutually exclusive and exhaustive events belonging to a sample space is called a partition of the sample space. With reference to any sample space S, events A and its complement (Δ) form a partition as they are mutually exclusive and their union is the entire sample space.
π Definition β Partition of the Sample Space: A set of events that are both mutually exclusive and collectively exhaustive.
Equally Likely Events
Two events A and B are said to be equally likely when one event is as likely to occur as the other. In other words, each event should occur in equal number in repeated trials. For example, when a fair coin is tossed, a head is as likely to appear as a tail, with a proportion of 1/2. If a card is drawn from a well-shuffled deck, each card is equally likely to be drawn, with a proportion of 1/52.
π Definition β Equally Likely Events: Events that have the same chance of occurring in a random experiment.
Subjective or Personalistic Probability
The subjective or personalistic probability is a measure of the strength of a personβs belief regarding the occurrence of an event A. Probability in this sense is purely subjective and is based on whatever evidence is available to the individual. Its disadvantage is that two or more persons faced with the same evidence may arrive at different probabilities. For example, in a trial, two judges might find the accused guilty while a third finds the evidence insufficient, illustrating the subjective nature of this probability.
π Definition β Subjective Probability: A measure of an individual's personal belief in the likelihood of an event, which can vary from person to person.
1. The Classical or βA Prioriβ Definition of Probability
If a random experiment can produce n mutually exclusive and equally likely outcomes, and if m out of these outcomes are considered favorable to the occurrence of a certain event A, then the probability of the event A, denoted by P(A), is defined as the ratio m/n.
π Formula: P(A) = m/n = Number of favorable outcomes / Total number of possible outcomes
This definition was formulated by the French mathematician P.S. Laplace and can be conveniently used in experiments where the total number of possible outcomes and the number of outcomes favorable to an event can be determined.
π Example-1: If a card is drawn from an ordinary deck of 52 playing cards, the probability that it is a red card is:
- Let A be the event that the card is red. Number of favorable outcomes = 26 (13 diamonds + 13 hearts).
- P(A) = 26/52 = 1/2. The probability that the card is a 10:
- Let B be the event that the card is a 10. Number of favorable outcomes = 4.
- P(B) = 4/52 = 1/13.
π Example-2: A fair coin is tossed three times. The sample space S = {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}, so n(S) = 8. Let A be the event of at least one head. Then A = {HHH, HHT, HTH, THH, HTT, THT, TTH}, so n(A) = 7.
- P(A) = 7/8.
π Example-3: Four items are taken at random from a box of 12 items (3 faulty, 9 good). The box is rejected if more than 1 item is faulty. Find the probability that the box is accepted. The total sample points are 12C4 = 495. The box is accepted if 0 or 1 faulty items are chosen.
- n(A) = (3C0 Γ 9C4) + (3C1 Γ 9C3) = 126 + 252 = 378.
- P(A) = 378/495 = 0.76 or 76%.
Shortcomings of the Classical Definition
- It involves circular reasoning because the term "equally likely" really means "equally probable," thus defining probability using concepts that presume a prior knowledge of the meaning of probability.
- This definition becomes vague when the possible outcomes are infinite or uncountable.
- This definition is not applicable when the assumption of equally likely does not hold, which occurs in numerous real-world situations.
The Relative Frequency Definition of Probability
The essence of this definition is that if an experiment is repeated a large number of times under identical conditions, and if the event of our interest occurs a certain number of times, then the proportion in which this event occurs is regarded as the probability of that event. For example, the proportion of students who obtain a first division in a large exam can be regarded as the probability of obtaining a first division.
π Definition β Relative Frequency Probability: The probability of an event is the proportion of times it occurs in a large number of repeated trials under identical conditions.
π‘ Why this matters: The relative frequency definition overcomes the limitations of the classical definition by being applicable to situations where outcomes are not equally likely or are infinite.
β Key Takeaways
This lecture introduces the foundational concepts for probability theory, including mutually exclusive, exhaustive, and equally likely events, which are essential for defining a sample space's partition. The classical definition, P(A) = m/n, is useful only when all outcomes are equally likely and finite, but it suffers from circular reasoning and inapplicability to many real-world scenarios. The relative frequency definition provides a practical alternative by defining probability as the proportion of occurrences in many trials, making it more widely applicable. The subjective probability approach acknowledges individual belief differences but lacks objectivity. Understanding these definitions and their limitations is crucial for selecting the correct approach in statistical problems.
π§ Quick Revision Questions
- What is meant by mutually exclusive events? Give an example.
- What is a partition of the sample space?
- State the classical definition of probability and write its formula.
- Explain one major shortcoming of the classical definition of probability.
- How does the relative frequency definition of probability differ from the classical definition?
π Lecture 19 β Relative Frequency Definition of Probability β’ Axiomatic Definition of Probability β’ Laws of Probability β’ Rule of Complementation β’ Addition Theorem
π Overview: This lecture presents multiple definitions of probability, including the relative frequency (a posteriori) and axiomatic definitions. It also introduces fundamental laws of probability, specifically the rule of complementation and the addition theorem, which are essential for solving probability problems.
ποΈ Topics Covered
The lecture covers the relative frequency definition of probability with empirical examples from coin-tossing and birth statistics, the axiomatic definition of probability based on Kolmogorov's three axioms, and two key laws of probability: the rule of complementation and the addition theorem for non-mutually exclusive events.
π Lecture Summary
The Relative Frequency Definition of Probability ('A Posteriori' Definition of Probability)
If a random experiment is repeated a large number of times, say n times, under identical conditions and if an event A is observed to occur m times, then the probability of the event A is defined as the limit of the relative frequency m/n as n tends to infinity. Symbolically, P(A) = Lim (m/n) as n β β. The definition assumes that as n increases indefinitely, the ratio m/n tends to become stable at the numerical value P(A). As its name suggests, the relative frequency definition relates to the relative frequency with which an event occurs in the long run. This definition is very useful in practical situations where the classical definition cannot be applied because various possible outcomes are NOT equally likely. This type of probability is also called empirical probability as it is based on EMPIRICAL evidence (observational data), and can also be called STATISTICAL PROBABILITY.
π Definition β Relative Frequency Definition of Probability: If a random experiment is repeated n times and event A occurs m times, then P(A) = limit of m/n as n approaches infinity. π Formula: P(A) = lim (m/n) as n β β β The probability of an event is the stable long-run proportion of times it occurs. π Example β Coin tossing by Kerrich in 1946: A coin was tossed 10,000 times, yielding 5067 heads and 4933 tails. The proportion of heads fluctuated widely at first (e.g., around 0.6 after 10 tosses) but settled down near 0.5 as the number of tosses increased to 10,000. This hypothetical limiting value of 0.5 is the statistical probability of heads.
π Example β Male birth proportions in England (1956): For major regions (each based on ~100,000 births), proportions of male births ranged between 0.512 and 0.517 (countrywide: 0.514). For rural districts of Dorset (each based on ~200 births), proportions fluctuated widely from 0.38 to 0.59. The larger sample size produces greater constancy, and the hypothetical limiting value is the statistical probability of a male birth (approximately 0.514).
π‘ Why this matters: The relative frequency definition is the foundation of mathematical statistics and is used when classical equally-likely assumptions do not hold. It relies on empirical data collected from repeated experiments.
The Axiomatic Definition of Probability
This definition, introduced in 1933 by the Russian mathematician Andrei N. Kolmogorov, is based on a set of AXIOMS. Let S be a sample space with sample points Eβ, Eβ, ... Eα΅’, ... Eβ. To each sample point, we assign a real number, denoted P(Eα΅’) and called the probability of Eα΅’, that must satisfy the following basic axioms:
Axiom 1: For any event Eα΅’, 0 β€ P(Eα΅’) β€ 1.
Axiom 2: P(S) = 1 for the sure event S.
Axiom 3: If A and B are mutually exclusive events (subsets of S), then P(A βͺ B) = P(A) + P(B).
According to the axiomatic theory, probability is defined as a non-negative real number attached to each sample point Eα΅’ such that the sum of all such numbers must equal ONE. The assignment of probabilities may be based on past evidence (empirical probability) or on underlying conditions ensuring equally likely outcomes (classical definition).
π Definition β Axiomatic Probability: Probability assignment to sample points satisfying three axioms: non-negativity (0 β€ P(Eα΅’) β€ 1), certainty (P(S) = 1), and additivity for mutually exclusive events (P(AβͺB) = P(A) + P(B)).
π Example β Births in England and Wales (1956): The table shows 716,740 total births. Relative frequencies (treated as probabilities): Male liveborn = 0.5021, Male stillborn = 0.0120, Female liveborn = 0.4750, Female stillborn = 0.0109. For compound events: P(Male birth M) = P(A or B) = P(A) + P(B) = 0.5021 + 0.0120 = 0.5141. P(Stillbirth S) = P(B or D) = P(B) + P(D) = 0.0120 + 0.0109 = 0.0229.
π‘ Why this matters: The axiomatic definition provides a formal mathematical framework for probability that unifies both classical and empirical approaches.
Law of Complementation
If Δ is the complement of an event A relative to the sample space S, then P(Δ) = 1 - P(A). Hence the probability of the complement of an event is equal to one minus the probability of the event. Complementary probabilities are very useful for solving questions like "What is the probability that, in tossing two fair dice, at least one even number will appear?"
π Definition β Rule of Complementation: P(Δ) = 1 - P(A), where Δ is the complement of event A in sample space S. π Formula: P(Δ) = 1 - P(A) β The probability an event does NOT occur is 1 minus the probability it does occur. π Example β A coin is tossed 4 times in succession: The sample space has 2β΄ = 16 equally likely sample points. Let A be the event "at least one head occurs." Its complement Δ is "no head" = {TTTT}. P(Δ) = 1/16. Therefore, P(A) = 1 - P(Δ) = 1 - 1/16 = 15/16.
Addition Law
If A and B are any two events defined in a sample space S, then P(A βͺ B) = P(A) + P(B) - P(A β© B). In words: "If two events A and B are not mutually exclusive, then the probability that at least one of them occurs is given by the sum of the separate probabilities of events A and B minus the probability of the joint event A β© B."
π Definition β Addition Theorem (General Addition Law): P(A βͺ B) = P(A) + P(B) - P(A β© B), for any two events A and B. π Formula: P(A βͺ B) = P(A) + P(B) - P(A β© B) β The probability of A or B occurring equals the sum of individual probabilities minus the probability of both occurring together. π Example β From the birth data: P(Male) = 0.5141, P(Stillbirth) = 0.0229, P(Male and Stillbirth) = 0.0120. Then P(Male or Stillbirth) = 0.5141 + 0.0229 - 0.0120 = 0.5250.
β Key Takeaways
The most critical points from this lecture are: (1) The relative frequency definition defines probability as the limit of m/n as n approaches infinity, making it useful for empirical situations where outcomes are not equally likely. (2) The axiomatic definition, based on Kolmogorov's three axioms, provides a formal mathematical foundation for all probability. (3) The rule of complementation P(Δ) = 1 - P(A) is extremely useful when the complement of an event is easier to compute, as shown in the coin-tossing example where P(at least one head) = 15/16. (4) The addition theorem P(AβͺB) = P(A) + P(B) - P(Aβ©B) accounts for double-counting when events are not mutually exclusive. (5) Inductive (subjective) probability is non-quantifiable and cannot be used in mathematical statistics, which relies on statistical probability.
π§ Quick Revision Questions
-
State the relative frequency definition of probability and explain why it is called "a posteriori" probability.
-
What are Kolmogorov's three axioms of probability? Explain each axiom in simple terms.
-
Using the rule of complementation, find the probability of getting at least one head when a coin is tossed 4 times. Show your work.
-
For the addition theorem, why must we subtract P(Aβ©B) when computing P(AβͺB)? Provide a simple intuitive explanation.
-
From the birth data in England and Wales (1956), if P(Male) = 0.5141, P(Stillbirth) = 0.0229, and P(Male and Stillbirth) = 0.0120, compute P(Male or Stillbirth) using the addition theorem.
π Lecture 20 β Application of Addition Theorem, Conditional Probability, Multiplication Theorem
π Overview: This lecture extends the foundations of probability by discussing the Addition Theorem in detail, including its application to both mutually exclusive and non-mutually exclusive events. It then introduces the crucial concept of conditional probability, which leads directly to the Multiplication Theorem for finding the probability of joint events. These theorems are essential for solving a wide range of practical probability problems.
ποΈ Topics Covered
The lecture begins by revisiting the Addition Law and demonstrating its application through examples involving cards and dice. It then covers corollaries for mutually exclusive events and uses a horse race example to illustrate probability for non-equally likely events. The concept of Conditional Probability is introduced, showing how additional information reduces the sample space. Finally, the Multiplication Theorem of probability is derived and applied to a problem involving defective and good items.
π Lecture Summary
ADDITION LAW
The Addition Law (General Addition Theorem) states that for any two events A and B defined in a sample space S, the probability that at least one of them occurs is given by: P(A βͺ B) = P(A) + P(B) β P(A β© B) . In words, it is the sum of the separate probabilities minus the probability of their joint occurrence. This law applies when events are not mutually exclusive.
π Definition β Addition Law: The rule that gives the probability of the union of two events, accounting for any overlap between them.
π Formula: P(A βͺ B) = P(A) + P(B) β P(A β© B) β The probability of either A or B (or both) is the sum of their individual probabilities minus the probability of them both happening.
π Example: Selecting one card from a deck of 52. Let A be the event "card is a club" and B be "card is a face card". Find P(A βͺ B) (a club or a face card or both).
- Step 1: Identify probabilities. P(A) = 13/52 (13 clubs). P(B) = 12/52 (12 face cards). P(A β© B) = 3/52 (3 face cards are clubs).
- Step 2: Apply the formula. P(A βͺ B) = 13/52 + 12/52 - 3/52 = 22/52.
COROLLARY-1
If A and B are mutually exclusive events (they cannot occur simultaneously, meaning A β© B = β ), then the Addition Law simplifies. Since P(A β© B) = 0, the formula becomes: P(A βͺ B) = P(A) + P(B) .
π Example: Tossing a pair of dice and being interested in the probability of getting a total of 5 (event A) or a total of 11 (event B). These events are mutually exclusive.
- Step 1: Find individual probabilities. P(A) = 4/36 (outcomes: (1,4), (2,3), (3,2), (4,1)). P(B) = 2/36 (outcomes: (5,6), (6,5)).
- Step 2: Apply formula for mutually exclusive events. P(A βͺ B) = 4/36 + 2/36 = 6/36 = 1/6 β 16.67%.
COROLLARY-2
If Aβ, Aβ, ..., Aβ are k mutually exclusive events, then the probability that one of them occurs is the sum of the probabilities of the separate events: P(Aβ βͺ Aβ βͺ ... βͺ Aβ) = P(Aβ) + P(Aβ) + ... + P(Aβ) .
π Example (Race Problem): Three horses A, B, C are in a race. A is twice as likely to win as B, and B is twice as likely to win as C. What is P(A or B wins)? The events are not equally likely.
- Step 1: Define probabilities relative to C. Let P(C) = p. Then P(B) = 2p and P(A) = 4p.
- Step 2: Use the fact that A, B, C are mutually exclusive and collectively exhaustive (sum = 1). So, p + 2p + 4p = 1 β 7p = 1 β p = 1/7.
- Step 3: Find specific probabilities. P(A) = 4/7, P(B) = 2/7, P(C) = 1/7.
- Step 4: Since A and B are mutually exclusive, P(A βͺ B) = P(A) + P(B) = 4/7 + 2/7 = 6/7.
π‘ Why this matters: This example shows how to solve problems when the events are not equally likely but their relationships are known.
CONDITIONAL PROBABILITY
Conditional probability is the probability of an event being calculated after some additional information is received, which reduces the original sample space. The conditional probability of event A given that event B has occurred (and P(B) > 0) is defined as: P(A|B) = P(A β© B) / P(B) . Similarly, P(B|A) = P(A β© B) / P(A) if P(A) > 0. P(A|B) satisfies all basic axioms of probability.
π Definition β Conditional Probability (P(A|B)): The probability that event A occurs given that event B has already occurred. It is valid only if P(B) > 0.
π Formula: P(A|B) = P(A β© B) / P(B) (where P(B) > 0) β The probability of A given B is the probability of both A and B happening divided by the probability of B.
π Example 1: Tossing a fair die. What is the probability of getting a 6 (event A) given that the die shows an even number (event B)?
- Step 1: The reduced sample space is B = {2, 4, 6}. There is one favorable outcome for A (6).
- Step 2: In the reduced sample space, P(A|B) = 1/3.
π Example 2: School records show 75% of students are from two-parent homes (B), and 20% are low-achievers from two-parent homes (A β© B). Find P(low achiever | two-parent home) = P(A|B).
- Step 1: Identify probabilities. P(B) = 0.75, P(A β© B) = 0.20.
- Step 2: Apply formula. P(A|B) = P(A β© B) / P(B) = 0.20 / 0.75 = 0.27.
MULTIPLICATION THEOREM OF PROBABILITY
The Multiplication Theorem is derived from the definition of conditional probability. By rearranging the formula P(A|B) = P(A β© B) / P(B), we get P(A β© B) = P(B) * P(A|B) . Similarly, P(A β© B) = P(A) * P(B|A) . This is the general rule of multiplication for any two events.
π Definition β Multiplication Law: The rule that the probability that two events A and B will both occur is equal to the probability of one event multiplied by the conditional probability of the other given that the first has occurred.
π Formulas: P(A β© B) = P(A) * P(B|A) (if P(A) > 0) ; P(A β© B) = P(B) * P(A|B) (if P(B) > 0) β The probability of both A and B is the probability of the first event times the probability of the second event after the first has happened.
π Example: A box has 15 items (11 good, 4 defective). Two items are selected without replacement. Find P(first is good (A) AND second is defective (B)).
- Step 1: Find P(A). P(A) = 11/15.
- Step 2: Given A occurred, 14 items remain (10 good, 4 defective). Find P(B|A). P(B|A) = 4/14.
- Step 3: Apply the multiplication theorem. P(A β© B) = P(A) * P(B|A) = (11/15) * (4/14) = 44/210 = 0.16.
π‘ Why this matters: The distinction between the two theorems is key: use the addition theorem for the probability of "either A or B" and the multiplication theorem for the probability of "both A and B."
β Key Takeaways
The most critical concepts from this lecture are the formulas and distinctions between the addition and multiplication theorems. You must memorize the general addition law P(AβͺB) = P(A) + P(B) β P(Aβ©B) and know when to simplify it to P(A) + P(B) for mutually exclusive events. The definition of conditional probability, P(A|B) = P(Aβ©B) / P(B), is fundamental, as it leads directly to the multiplication law P(Aβ©B) = P(A) * P(B|A). Finally, remember that the addition theorem answers "or" questions, while the multiplication theorem answers "and" questions, and both are applied based on the specific conditions and sample space given.
π§ Quick Revision Questions
- What is the general formula for the Addition Law of probability, and when can it be simplified?
- Define conditional probability P(A|B) in terms of a joint and a marginal probability. What condition is required for it to be valid?
- Write down the two forms of the general Multiplication Theorem.
- In the "Three horses" example, why couldn't the probabilities be set directly to 1/3 each? How were they derived instead?
- A bag contains 5 red and 3 blue marbles. Two marbles are drawn without replacement. What is the probability the first is red and the second is blue?
π Lecture 21 β Independent and Dependent Events
π Overview: This lecture introduces the critical distinction between independent and dependent events in probability theory. It demonstrates how to determine independence through the Multiplication Theorem and illustrates the practical application of marginal probability using real-world birth statistics.
ποΈ Topics Covered
The lecture begins with an extended example applying both Addition and Multiplication Theorems of probability, then formally defines independent events and the special case of the Multiplication Theorem. It provides a detailed example with dice to demonstrate independence, examines a real birth statistics table to determine dependence between sex and stillbirth status, and concludes with the definition and computation of marginal probabilities from joint probability tables.
π Lecture Summary
Example Illustrating Addition and Multiplication Theorems
A bag contains 10 white and 3 black balls. Another bag contains 3 white and 5 black balls. Two balls are transferred from first bag and placed in the second, and then one ball is taken from the latter. We need the probability that it is a white ball.
Let A represent the event that 2 balls are drawn from the first bag and transferred to the second bag. A can occur in three mutually exclusive ways:
- Aβ = 2 white balls transferred
- Aβ = 1 white ball and 1 black ball transferred
- Aβ = 2 black balls transferred
P(Aβ) = C(10,2)/C(13,2) = 45/78 P(Aβ) = C(10,1)ΓC(3,1)/C(13,2) = 30/78 P(Aβ) = C(3,2)/C(13,2) = 3/78
After transferring 2 balls, the second bag contains: i) If 2 white balls transferred: 5 white, 5 black β P(W/Aβ) = 5/10 ii) If 1 white and 1 black transferred: 4 white, 6 black β P(W/Aβ) = 4/10 iii) If 2 black balls transferred: 3 white, 7 black β P(W/Aβ) = 3/10
Let W represent the event that a white ball is drawn from the second bag. P(W) = P(Aββ©W) + P(Aββ©W) + P(Aββ©W) P(Aββ©W) = P(Aβ)P(W/Aβ) = 45/78 Γ 5/10 = 15/52 P(Aββ©W) = P(Aβ)P(W/Aβ) = 30/78 Γ 4/10 = 2/13 P(Aββ©W) = P(Aβ)P(W/Aβ) = 3/78 Γ 3/10 = 3/260
P(W) = 15/52 + 2/13 + 3/260 = 59/130 = 0.45
Independent Events
Two events A and B in the same sample space S are defined to be independent (or statistically independent) if the probability that one event occurs is not affected by whether the other event has or has not occurred. That is: P(A/B) = P(A) and P(B/A) = P(B)
π Definition β Independent Events: Two events A and B are independent if and only if P(Aβ©B) = P(A)P(B). This is known as the special case of the Multiplication Theorem of Probability.
Rationale
According to the multiplication theorem of probability: P(Aβ©B) = P(A)P(B/A). Putting P(B/A) = P(B), we obtain P(Aβ©B) = P(A)P(B).
The events A and B are defined to be dependent if P(Aβ©B) β P(A) Γ P(B). This means the occurrence of one event in some way affects the probability of the other.
π‘ Why this matters: Two events that are independent can never be mutually exclusive.
Example: Demonstrating Independence with Dice
Two fair dice, one red and one green, are thrown. Let A denote the event that the red die shows an even number. Let B denote the event that the green die shows a 5 or a 6.
The sample space S has 36 outcomes. P(A) = 3/6, P(B) = 2/6. The joint event Aβ©B contains 6 outcomes: (2,5), (4,5), (6,5), (2,6), (4,6), (6,6). P(Aβ©B) = 6/36.
P(A)P(B) = 3/6 Γ 2/6 = 6/36 = P(Aβ©B)
Therefore, events A and B are independent.
Example: Sex and Stillbirth Dependence
Table of births in England and Wales in 1956 (proportions):
| Liveborn | Stillborn | Total | |
|---|---|---|---|
| Male | .5021 | .0120 | .5141 |
| Female | .4750 | .0109 | .4859 |
| Total | .9771 | .0229 | 1.0000 |
Let M = male birth, S = stillbirth.
Conditional probability of stillbirth given male birth = P(S/M) = 8609/368490 = 0.0234 Conditional probability of stillbirth given female birth = 7796/348258 = 0.0224 Overall (unconditional) proportion of stillbirths = 16405/716740 = 0.0229
The conditional probability of stillbirth among boys is slightly higher than the overall proportion, while among girls it is slightly lower. Therefore, sex and stillbirth are statistically dependent β the sex of a baby affects its chance of being stillborn.
Marginal Probability
The probabilities that appear in the margins of a table are known as Marginal Probabilities.
π Definition β Marginal Probability: A probability obtained by summing joint probabilities in a row or column of a contingency table.
From the table above:
- P(male birth) = 0.5141 (appears in the margin)
- P(female birth) = 0.4859
- P(live birth) = 0.9771
- P(stillbirth) = 0.0229
P(male birth) = P(male live-born) + P(male stillborn) = 0.5021 + 0.0120 = 0.5141
This follows the Addition Theorem of Probability for mutually exclusive events β joint probabilities in any row add up to yield the corresponding marginal probability.
π Formula: P(marginal event) = Ξ£ joint probabilities in that row (or column)
Example: Conditional Probability Calculation
P(stillbirth/male birth) = P(male birth and stillbirth)/P(male birth) = 0.0120/0.5141 = 0.0233
β Key Takeaways
The Multiplication Theorem for independent events states that P(Aβ©B) = P(A)P(B), which is the essential test for independence. Two events are independent if the occurrence of one does not affect the probability of the other; they are dependent otherwise. Importantly, independent events can never be mutually exclusive. Marginal probabilities are found by summing joint probabilities across rows or columns of a contingency table, and they represent unconditional probabilities of individual events. Conditional probabilities can be compared to marginal probabilities to determine independence β if P(A/B) β P(A), the events are dependent.
π§ Quick Revision Questions
- What is the mathematical condition that must hold for two events A and B to be considered independent?
- In the bag transfer example, what was the final probability of drawing a white ball from the second bag, and why was the Multiplication Theorem needed?
- Why can two independent events never be mutually exclusive? Explain the logical reasoning.
- In the birth statistics example, how did comparing conditional probabilities to the overall proportion reveal dependence between sex and stillbirth?
- How is a marginal probability computed from a table of joint probabilities, and which probability theorem justifies this computation?