π Lecture 29 β Permutations and Combinations
π Overview: This lecture introduces fundamental counting rules used in probability calculations, specifically permutations and combinations. It then transitions into the foundational concepts of probability theory, including random experiments, sample spaces, and different types of events. Understanding these concepts is crucial for defining and calculating probabilities in various real-world and mathematical scenarios.
ποΈ Topics Covered
This lecture covers the rule of permutation for ordered selections, including factorials and permutations of non-distinct objects. It then explains the rule of combination for unordered selections and its properties as a binomial coefficient. The second half of the lecture introduces random experiments, sample spaces, and various types of events such as simple, compound, complementary, mutually exclusive, exhaustive, and equally likely events.
π Lecture Summary
COUNTING RULES:
As discussed in the last lecture, there are certain rules that facilitate the calculations of probabilities in certain situations. They are known as counting rules and include concepts of: 1) Multiple Choice, 2) Permutations, and 3) Combinations. We have already discussed the rule of multiplication in the last lecture.
RULE OF PERMUTATION:
A permutation is any ordered subset from a set of n distinct objects. For example, if we have the set {a, b}, then one permutation is ab, and the other permutation is ba. The number of permutations of r objects, selected in a definite order from n distinct objects is denoted by the symbol nPr, and is given by nPr = n (n β 1) (n β 2) β¦(n β r + 1) = n! / (n β r)!.
π Definition β Permutation: an ordered subset from a set of n distinct objects. π Formula: nPr = n! / (n β r)! β The number of ways to select and arrange r items in order from a set of n distinct items. π Example: A club consists of four members. How many ways are there of selecting three officers: president, secretary and treasurer? It is evident that the order in which 3 officers are to be chosen, is of significance. Thus there are 4 choices for the first office, 3 choices for the second office, and 2 choices for the third office. Hence the total number of ways in which the three offices can be filled is 4 Γ 3 Γ 2 = 24. The same result is obtained by applying the rule of permutations: 4P3 = 4! / (4 - 3)! = 4 Γ 3 Γ 2 = 24.
FACTORIALS
7! = 7 Γ 6 Γ 5 Γ 4 Γ 3 Γ 2 Γ 1. 6! = 6 Γ 5 Γ 4 Γ 3 Γ 2 Γ 1. 1! = 1. Also, we define 0! = 1.
The factorials and permutations are used to describe possible arrangements of distinct objects. A tree diagram can provide an organized way of listing possible arrangements for a permutation problem.
PERMUTATIONS:
In the formula of nPr, if we put r = n, we obtain: nPn = n(n β 1) (n β 2) β¦ 3 Γ 2 Γ 1 = n!. I.e. the total number of permutations of n distinct objects, taking all n at a time, is equal to n!.
π Definition β The total number of permutations of n distinct objects, taking all n at a time, is equal to n!. π Example: Suppose that there are three persons A, B & D, and that they wish to have a photograph taken. The total number of ways in which they can be seated on three chairs (placed side by side) is 3P3 = 3! = 6. These are: ABD, ADB, BAD, BDA, DAB, & DBA.
The above discussion pertained to the case when all the objects under consideration are distinct objects. If some of the objects are not distinct, the formula of permutations modifies as given below: The number of permutations of n objects, selected all at a time, when n objects consist of n1 of one kind, n2 of a second kind, β¦, nk of a kth kind, is P = n! / (n1! n2! β¦.. nk!), where Ξ£ ni = n.
π Formula: Permutations of non-distinct objects: P = n! / (n1! n2! β¦.. nk!), where Ξ£ ni = n. π Example: How many different (meaningless) words can be formed from the word βcommitteeβ? In this example: n = 9 (because the total number of letters in this word is 9), n1 = 1 (c), n2 = 1 (o), n3 = 2 (mβs), n4 = 1 (i), n5 = 2 (tβs), n6 = 2 (eβs). Hence, the total number of (meaningless) words (permutations) is: P = 9! / (1! 1! 2! 1! 2! 2!) = (9 Γ 8 Γ 7 Γ 6 Γ 5 Γ 4 Γ 3 Γ 2 Γ 1) / (1 Γ 1 Γ 2 Γ 1 Γ 1 Γ 2 Γ 1 Γ 2 Γ 1) = 45360.
RULE OF COMBINATION:
A combination is any subset of r objects, selected without regard to their order, from a set of n distinct objects. The total number of such combinations is denoted by the symbol nCr or (n choose r), and is given by (n choose r) = n! / [r! (n β r)!], where r < n.
It should be noted that nPr = r! * (n choose r). In other words, every combination of r objects (out of n objects) generates r! permutations.
π Definition β Combination: any subset of r objects, selected without regard to their order, from a set of n distinct objects. π Formula: (n choose r) = n! / [r! (n β r)!], where r < n. π Example: Suppose we have a group of three persons, A, B, & C. If we wish to select a group of two persons out of these three, the three possible groups are {A, B}, {A, C} and {B, C}. In other words, the total number of combinations of size two out of this set of size three is 3. Now, suppose that our interest lies in forming a committee of two persons, one of whom is to be the president and the other the secretary of a club. The six possible committees are: (A, B), (B, A), (A, C), (C, A), (B, C) & (C, B). In other words, the total number of permutations of two persons out of three is 6. And the point to note is that each of three combinations mentioned earlier generates 2 = 2! permutations.
The quantity (n choose r) or nCr is also called a binomial co-efficient because of its appearance in the binomial expansion of (a + b)^n = Ξ£ (n choose r) a^(n-r) b^r. The binomial co-efficient has two important properties: i) (n choose r) = (n choose n-r), and ii) (n choose n-r) + (n choose r) = (n+1 choose r). Also, it should be noted that (n choose 0) = 1 = (n choose n) and (n choose 1) = n = (n choose n-1).
π Example: A three-person committee is to be formed out of a group of ten persons. In how many ways can this be done? Since the order in which the three persons of the committee are chosen, is unimportant, it is therefore an example of a problem involving combinations. Thus the desired number of combinations is (10 choose 3) = 10! / (3! 7!) = (10 Γ 9 Γ 8 Γ 7 Γ 6 Γ 5 Γ 4 Γ 3 Γ 2 Γ 1) / (3 Γ 2 Γ 1 Γ 7 Γ 6 Γ 5 Γ 4 Γ 3 Γ 2 Γ 1) = 120. In other words, there are one hundred and twenty different ways of forming a three-person committee out of a group of only ten persons!
π Example: In how many ways can a person draw a hand of 5 cards from a well-shuffled ordinary deck of 52 cards? The total number of ways of doing so is given by (52 choose 5) = (52 Γ 51 Γ 50 Γ 49 Γ 48) / (5 Γ 4 Γ 3 Γ 2 Γ 1) = 2,598,960.
Having reviewed the counting rules that facilitate calculations of probabilities in a number of problems, let us now begin the discussion of concepts that lead to the formal definitions of probability. The first concept in this regard is the concept of Random Experiment. The term experiment means a planned activity or process whose results yield a set of data. A single performance of an experiment is called a trial. The result obtained from an experiment or a trial is called an outcome.
RANDOM EXPERIMENT:
An experiment which produces different results even though it is repeated a large number of times under essentially similar conditions is called a Random Experiment. The tossing of a fair coin, the throwing of a balanced die, drawing of a card from a well-shuffled deck of 52 playing cards, selecting a sample, etc. are examples of random experiments.
A random experiment has three properties: i) The experiment can be repeated, practically or theoretically, any number of times. ii) The experiment always has two or more possible outcomes. An experiment that has only one possible outcome, is not a random experiment. iii) The outcome of each repetition is unpredictable, i.e. it has some degree of uncertainty. π‘ Why this matters: These three properties help identify whether a process qualifies as a random experiment, which is the basis for studying probability. For example, interviewing a person to find out whether or not he or she is a smoker is an example of a random experiment because it fulfills all three properties.
SAMPLE SPACE:
A set consisting of all possible outcomes that can result from a random experiment (real or conceptual), can be defined as the sample space for the experiment and is denoted by the letter S. Each possible outcome is a member of the sample space, and is called a sample point in that space.
π Example-1: The experiment of tossing a coin results in either of the two possible outcomes: a head (H) or a tail (T). The sample space for this experiment may be expressed in set notation as S = {H, T}. βHβ and βTβ are the two sample points. π Example-2: The sample space for tossing two coins once (or tossing a coin twice) will contain four possible outcomes denoted by S = {HH, HT, TH, TT}. π Example-3: The sample space S for the random experiment of throwing two six-sided dice can be described by the Cartesian product A Γ A, where A = {1, 2, 3, 4, 5, 6}. In other words, S = A Γ A = {(x, y) | x β A and y β A}, where x denotes the number of dots on the upper face of the first die, and y denotes the number of dots on the upper face of the second die. Hence, S contains 36 outcomes or sample points.
EVENTS:
Any subset of a sample space S of a random experiment, is called an event. In other words, an event is an individual outcome or any number of outcomes (sample points) of a random experiment.
SIMPLE & COMPOUND EVENTS: An event that contains exactly one sample point, is defined as a simple event. A compound event contains more than one sample point, and is produced by the union of simple events. π Example: The occurrence of a 6 when a die is thrown, is a simple event, while the occurrence of a sum of 10 with a pair of dice, is a compound event, as it can be decomposed into three simple events (4, 6), (5, 5) and (6, 4).
OCCURRENCE OF AN EVENT: An event A is said to occur if and only if the outcome of the experiment corresponds to some element of A.
COMPLEMENTARY EVENT: The event βnot-Aβ is denoted by Δ or A^c and called the negation (or complementary event) of A. π Example: If we toss a coin once, then the complement of βheadsβ is βtailsβ. If we toss a coin four times, then the complement of βat least one headβ is βno headsβ.
A sample space consisting of n sample points can produce 2^n different subsets (or simple and compound events). π Example: Consider a sample space S containing 3 sample points, i.e. S = {a, b, c}. Then the 2^3 = 8 possible subsets are Ο, {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c}. Each of these subsets is an event. The subset {a, b, c} is the sample space itself and is also an event. It always occurs and is known as the certain or sure event. The empty set Ο is also an event, sometimes known as impossible event, because it can never occur.
MUTUALLY EXCLUSIVE EVENTS:
Two events A and B of a single experiment are said to be mutually exclusive or disjoint if and only if they cannot both occur at the same time i.e. they have no points in common. π Example-1: When we toss a coin, we get either a head or a tail, but not both at the same time. The two events head and tail are therefore mutually exclusive. π Example-2: When a die is rolled, the events βeven numberβ and βodd numberβ are mutually exclusive as we can get either an even number or an odd number in one throw, not both at the same time. Three or more events originating from the same experiment are mutually exclusive if pair wise they are mutually exclusive. If the two events can occur at the same time, they are not mutually exclusive, e.g., if we draw a card from an ordinary deck of 52 playing cars, it can be both a king and a diamond. Therefore, kings and diamonds are not mutually exclusive.
EXHAUSTIVE EVENTS:
Events are said to be collectively exhaustive, when the union of mutually exclusive events is equal to the entire sample space S. π Example: In the coin-tossing experiment, βheadβ and βtailβ are collectively exhaustive events. In the die-tossing experiment, βeven numberβ and βodd numberβ are collectively exhaustive events.
PARTITION OF THE SAMPLE SPACE: A group of mutually exclusive and exhaustive events belonging to a sample space is called a partition of the sample space. With reference to any sample space S, events A and Δ form a partition as they are mutually exclusive and their union is the entire sample space.
EQUALLY LIKELY EVENTS:
Two events A and B are said to be equally likely, when one event is as likely to occur as the other. In other words, each event should occur in equal number in repeated trials. π Example: When a fair coin is tossed, the head is as likely to appear as the tail, and the proportion of times each side is expected to appear is 1/2. If a card is drawn out of a deck of well-shuffled cards, each card is equally likely to be drawn, and the probability that any card will be drawn is 1/52.
β Key Takeaways
Students must remember the distinction between permutations (ordered arrangements) and combinations (unordered selections), as well as their respective formulas: nPr = n!/(n-r)! and nCr = n!/[r!(n-r)!]. A random experiment has three essential properties: repeatability, multiple possible outcomes, and unpredictable results. The sample space contains all possible outcomes, and any subset of it is an event. Mutually exclusive events cannot occur simultaneously, exhaustive events cover the entire sample space, and equally likely events have the same chance of occurring.
π§ Quick Revision Questions
- A club has 10 members. How many different ways can they select a president, vice president, and secretary (order matters)?
- From a group of 8 people, how many different committees of 3 can be formed?
- What are the three properties that define a random experiment?
- When rolling a single six-sided die, what is the sample space? Give an example of a simple event and a compound event from this space.
- Are the events βdrawing a heartβ and βdrawing a kingβ from a standard deck of cards mutually exclusive? Why or why not?
π Lecture 30 β Definitions of Probability
π Overview: This lecture introduces the fundamental concept of probability, beginning with essential preliminary concepts such as mutually exclusive, exhaustive, and equally likely events. It then presents the two main approaches to probability: the subjective approach and the objective approach, with detailed coverage of the classical (a priori) definition and its limitations, leading to the relative frequency definition of probability.
ποΈ Topics Covered
The lecture covers mutually exclusive events with examples from coin tossing and dice rolling, exhaustive events and the partition of the sample space, equally likely events, the subjective or personalistic approach to probability, the classical definition of probability with several worked examples including card drawing and coin tossing, the shortcomings of the classical definition, and finally the relative frequency definition of probability.
π Lecture Summary
Mutually Exclusive Events
Mutually exclusive or disjoint events are two events A and B of a single experiment that cannot both occur at the same time, meaning they have no points in common. When we toss a coin, we get either a head or a tail, but not both at the same time, so the two events head and tail are mutually exclusive. When a die is rolled, the events 'even number' and 'odd number' are mutually exclusive as we can get either an even number or an odd number in one throw, not both at the same time. Three or more events are mutually exclusive if pairwise they are mutually exclusive. If the two events can occur at the same time, they are not mutually exclusive β for example, if we draw a card from an ordinary deck of 52 playing cards, it can be both a king and a diamond, so kings and diamonds are not mutually exclusive.
π Definition β Mutually Exclusive Events: Two events A and B are mutually exclusive if they cannot both occur at the same time, i.e., A β© B = β .
Exhaustive Events
Events are said to be collectively exhaustive when the union of mutually exclusive events is equal to the entire sample space S. In the coin-tossing experiment, 'head' and 'tail' are collectively exhaustive events. In the die-tossing experiment, 'even number' and 'odd number' are collectively exhaustive events.
π Definition β Exhaustive Events: A set of events is collectively exhaustive if their union equals the entire sample space S.
Partition of the Sample Space
A group of mutually exclusive and exhaustive events belonging to a sample space is called a partition of the sample space. With reference to any sample space S, events A and Δ form a partition as they are mutually exclusive and their union is the entire sample space.
Equally Likely Events
Two events A and B are said to be equally likely when one event is as likely to occur as the other. In other words, each event should occur in equal number in repeated trials. When a fair coin is tossed, the head is as likely to appear as the tail, and the proportion of times each side is expected to appear is 1/2. If a card is drawn out of a deck of well-shuffled cards, each card is equally likely to be drawn, and the proportion of times each card can be expected to be drawn in a very large number of draws is 1/52.
π Definition β Equally Likely Events: Two events A and B are equally likely when each event occurs in equal number in repeated trials.
Subjective or Personalistic Probability
The subjective or personalistic probability is a measure of the strength of a person's belief regarding the occurrence of an event A. Probability in this sense is purely subjective and is based on whatever evidence is available to the individual. It has a disadvantage that two or more persons faced with the same evidence may arrive at different probabilities. For example, suppose that a panel of three judges is hearing a trial. It is possible that, based on the evidence that is presented, two of them arrive at the conclusion that the accused is guilty while one of them decides that the evidence is NOT strong enough to draw this conclusion.
π Definition β Subjective Probability: A measure of the strength of a person's belief regarding the occurrence of an event, based on whatever evidence is available to the individual.
Classical Definition of Probability
The Classical or 'A Priori' Definition of Probability states: If a random experiment can produce n mutually exclusive and equally likely outcomes, and if m out of these outcomes are considered favorable to the occurrence of a certain event A, then the probability of the event A, denoted by P(A), is defined as the ratio m/n.
π Formula: P(A) = m/n = (Number of favourable outcomes)/(Total number of possible outcomes)
Example 1: If a card is drawn from an ordinary deck of 52 playing cards, find the probability that (i) the card is a red card, (ii) the card is a 10.
Solution: The total number of possible outcomes is 52, and we assume all possible outcomes are equally likely.
(i) Let A represent the event that the card drawn is a red card. The number of outcomes favourable to event A is 26 (13 diamonds + 13 hearts). Hence P(A) = 26/52 = 1/2.
(ii) Let B represent the event that the card drawn is a 10. There are 4 tens (one in each suit). Hence P(B) = 4/52 = 1/13.
Example 2: A fair coin is tossed three times. What is the probability that at least one head appears?
Solution: The sample space is S = {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}, so n(S) = 8. Let A denote the event that at least one head appears. Then A = {HHH, HHT, HTH, THH, HTT, THT, TTH} and n(A) = 7. Hence P(A) = n(A)/n(S) = 7/8.
Example 3: Four items are taken at random from a box of 12 items and inspected. The box is rejected if more than 1 item is found to be faulty. If there are 3 faulty items in the box, find the probability that the box is accepted.
Solution: The sample space S contains C(12,4) = 495 sample points. The box contains 3 faulty and 9 good items. The box is accepted if there is (i) no faulty items, or (ii) one faulty item in the sample of 4 items selected.
Let A denote the event the number of faulty items chosen is 0 or 1. Then: n(A) = C(3,0) Γ C(9,4) + C(3,1) Γ C(9,3) = 126 + 252 = 378 sample points.
Therefore, P(A) = 378/495 = 0.76.
Hence, the probability that the box is accepted is 76% (in spite of the fact that the box contains 3 faulty items).
Shortcomings of the Classical Definition
The classical definition has the following shortcomings: i) This definition is said to involve circular reasoning as the term "equally likely" really means "equally probable". Thus probability is defined by introducing concepts that presume a prior knowledge of the meaning of probability. ii) This definition becomes vague when the possible outcomes are infinite in number, or uncountable. iii) This definition is NOT applicable when the assumption of equally likely does not hold. And the fact of the matter is that there are numerous situations where the assumption of equally likely cannot hold.
Relative Frequency Definition of Probability
The relative frequency definition of probability states: If an experiment is repeated a large number of times under (more or less) identical conditions, and if the event of our interest occurs a certain number of times, then the proportion in which this event occurs is regarded as the probability of that event.
For example, a large number of students sit for the matric examination every year. Since the total number of students appearing is very large, the proportion of students who obtain the first division can be regarded as the probability of obtaining the first division, and the proportion of students who obtain the second division can be regarded as the probability of obtaining the second division, and so on.
π‘ Why this matters: The relative frequency definition overcomes the limitations of the classical definition by not requiring the assumption of equally likely outcomes, making it applicable to real-world situations where outcomes are not equally probable.
β Key Takeaways
The three essential preliminary concepts for understanding probability are mutually exclusive events (cannot occur together), exhaustive events (union equals sample space), and equally likely events (same chance of occurrence). The subjective approach to probability depends on personal belief and can vary between individuals. The classical definition P(A) = m/n requires mutually exclusive and equally likely outcomes, but suffers from circular reasoning and inapplicability when assumptions fail. The relative frequency definition overcomes these limitations by basing probability on long-run proportions from repeated experiments.
π§ Quick Revision Questions
- What are mutually exclusive events? Give an example involving a deck of cards.
- What is meant by a partition of the sample space?
- State the classical definition of probability and write its formula.
- What are the three main shortcomings of the classical definition of probability?
- How does the relative frequency definition of probability overcome the limitations of the classical definition?
π Lecture 31 β Relative Frequency, Axiomatic Definition & Laws of Probability
π Overview: This lecture introduces the relative frequency (empirical) definition of probability, which is based on the long-run behavior of events observed in repeated experiments. It then presents the formal axiomatic definition of probability, followed by key laws including the rule of complementation and the addition theorem. These concepts are essential for understanding how probability is quantified and applied in real-world situations where outcomes are not equally likely.
ποΈ Topics Covered
This lecture covers three main definitions of probability: the relative frequency ('a posteriori') definition based on the limit of relative frequency as the number of trials increases, the axiomatic definition based on Kolmogorov's three axioms, and the classical ('a priori') definition. It also discusses non-quantifiable (inductive) probability and presents the rule of complementation and the addition theorem for non-mutually exclusive events, illustrated with examples from coin-tossing and birth statistics.
π Lecture Summary
The Relative Frequency Definition of Probability (βA Posterioriβ Definition of Probability)
If a random experiment is repeated a large number of times, say n times, under identical conditions, and if an event A is observed to occur m times, then the probability of event A is defined as the limit of the relative frequency m/n as n tends to infinity. Symbolically, P(A) = Lim (m/n) as nββ. This definition assumes that as n increases indefinitely, the ratio m/n stabilizes at the numerical value P(A).
This definition is also called empirical probability because it is based on empirical evidence (observational data), or statistical probability, as it forms the basis of mathematical statistics. It is very useful in practical situations where the classical definition cannot be applied because outcomes are NOT equally likely.
π Definition β Relative Frequency Definition of Probability: The probability of an event A is the limit of the relative frequency (m/n) as the number of trials (n) approaches infinity. π Formula: P(A) = Lim (m/n) as nββ β The probability equals the long-run proportion of times the event occurs.
π Example-1 β Coin-Tossing (Kerrich, 1946): A coin was tossed 10,000 times, yielding 5067 heads and 4933 tails. The proportion of heads fluctuated widely at first but settled down near 0.5 as the number of tosses increased. The hypothetical limiting value (near 0.5) is the statistical probability of heads.
π Example-2 β Sex Ratio at Birth: Laplace recorded that among 215,599 births in France (1800-1802), there were 110,312 boys and 105,287 girls, giving proportions of 0.512 and 0.488. For major regions of England (each based on ~100,000 births), the proportion of male births ranged between 0.512 and 0.517 only. However, for rural districts of Dorset (each based on ~200 births), the proportion fluctuated between 0.38 and 0.59. The larger sample size leads to greater constancy. The hypothetical limiting value (near 0.514) is the statistical probability of a male birth.
π‘ Why this matters: The relative frequency definition applies when we need to compute probability numerically but the classical definition cannot be used because outcomes are not equally likely.
Non-Quantifiable (Inductive) Probability
This refers to the degree of belief which it is reasonable to place in a proposition on given evidence. It is difficult to express inductive probabilities numerically. Most statisticians conclude that inductive probability cannot be measured and therefore cannot be used in the mathematical theory of statistics. For this reason, usual methods of statistical inference (tests of significance, confidence intervals) are based entirely on the concept of statistical probability.
The Axiomatic Definition of Probability
Introduced in 1933 by the Russian mathematician Andrei N. Kolmogorov, this definition is based on a set of axioms. Let S be a sample space with sample points E1, E2, ... En. To each sample point, we assign a real number P(Ei), called the probability of Ei, that must satisfy the following basic axioms:
Axiom 1: For any event Ei, 0 < P(Ei) < 1. Axiom 2: P(S) = 1 for the sure event S. Axiom 3: If A and B are mutually exclusive events (subsets of S), then P(A βͺ B) = P(A) + P(B).
According to the axiomatic theory, some probability defined as a non-negative real number is attached to each sample point Ei such that the sum of all such numbers must equal ONE. The assignment of probabilities may be based on past evidence (empirical probability) or on underlying conditions ensuring equally likely outcomes (classical probability).
π Definition β Axiomatic Definition of Probability: A set of axioms (Kolmogorov's axioms) that any valid probability assignment must satisfy: non-negativity, the probability of the sample space equals 1, and additivity for mutually exclusive events.
π Example β Births in England and Wales (1956): From Table-1 data (716,740 births), relative frequencies are computed as probabilities in Table-2. For example, the probability of a male birth, P(M) = P(Male livebirth or Male stillbirth) = P(A) + P(B) = .5021 + .0120 = .5141. Similarly, P(Stillbirth) = P(B or D) = .0120 + .0109 = .0229.
Law of Complementation
If Δ is the complement of an event A relative to the sample space S, then P(Δ) = 1 - P(A). Hence, the probability of the complement of an event is equal to one minus the probability of the event. Complementary probabilities are very useful for questions like "What is the probability that, in tossing two fair dice, at least one even number will appear?"
π Definition β Rule of Complementation: The probability that an event does NOT occur equals 1 minus the probability that it does occur. π Formula: P(Δ) = 1 - P(A)
π Example β Coin tossed 4 times: What is the probability that at least one head occurs? The sample space has 2β΄ = 16 equally likely outcomes. Instead of finding P(at least one head) directly, find P(no head) = P(Δ) where Δ = {TTTT}. P(Δ) = 1/16. Then P(A) = 1 - P(Δ) = 1 - 1/16 = 15/16.
Addition Law (General Addition Theorem of Probability)
If A and B are any two events defined in a sample space S, then P(A βͺ B) = P(A) + P(B) - P(A β© B). In words: "If two events A and B are not mutually exclusive, then the probability that at least one of them occurs is given by the sum of the separate probabilities of events A and B minus the probability of the joint event A β© B."
π Definition β Addition Theorem: For any two events A and B, the probability of A or B occurring equals the sum of their individual probabilities minus the probability of both occurring. π Formula: P(A βͺ B) = P(A) + P(B) - P(A β© B)
β Key Takeaways
The relative frequency definition of probability defines probability as the limiting value of the relative frequency as the number of trials approaches infinity, making it applicable to real-world situations with unequally likely outcomes. The axiomatic definition, based on Kolmogorov's three axioms, provides the formal mathematical foundation for probability theory. The rule of complementation (P(Δ) = 1 - P(A)) simplifies probability calculations, especially for "at least one" problems. The addition theorem (P(A βͺ B) = P(A) + P(B) - P(A β© B)) accounts for overlapping events. Non-quantifiable inductive probability, based on degree of belief, is not used in mathematical statistics, which relies on statistical probability instead.
π§ Quick Revision Questions
- What is the relative frequency definition of probability, and what condition must be satisfied as n increases?
- State Kolmogorov's three axioms of probability.
- Using the law of complementation, if P(no head in 4 coin tosses) = 1/16, what is P(at least one head)?
- Write the addition theorem formula for two events A and B that are not mutually exclusive.
- Why is inductive (non-quantifiable) probability generally not used in mathematical statistics?
π Lecture 32 β Independent and Dependent Events, Multiplication Theorem & Marginal Probability
π Overview: This lecture explores the distinction between independent and dependent events in probability, introduces the multiplication theorem for independent events, and explains the concept of marginal probability. It provides practical examples to demonstrate how to determine event independence and calculate joint, conditional, and marginal probabilities, which are fundamental for statistical analysis.
ποΈ Topics Covered
The lecture begins by reviewing the addition and multiplication theorems through a comprehensive example involving ball transfers between bags. It then defines independent and dependent events, presents the special case of the multiplication theorem for independent events, and proves this theorem. Using dice and birth statistics examples, the lecture demonstrates how to verify independence and introduces marginal probability by examining how joint probabilities sum to marginal values in contingency tables.
π Lecture Summary
Review of Addition and Multiplication Theorems
The lecture opens with a comprehensive example involving two bags of colored balls. Bag A contains 10 white and 3 black balls, while Bag B contains 3 white and 5 black balls. Two balls are transferred from Bag A to Bag B, then one ball is drawn from Bag B. This problem requires both the addition and multiplication theorems to find the probability that the final ball is white.
The three mutually exclusive ways for the transfer event A to occur are: A1 (2 white balls transferred), A2 (1 white and 1 black ball transferred), and A3 (2 black balls transferred). Their probabilities are calculated as:
- P(A1) = C(10,2) Γ· C(13,2) = 45/78
- P(A2) = C(10,1)ΓC(3,1) Γ· C(13,2) = 30/78
- P(A3) = C(3,2) Γ· C(13,2) = 3/78
After each transfer scenario, the composition of Bag B changes, affecting the conditional probability of drawing white:
- After A1: Bag B has 5 white, 5 black β P(W/A1) = 5/10
- After A2: Bag B has 4 white, 6 black β P(W/A2) = 4/10
- After A3: Bag B has 3 white, 7 black β P(W/A3) = 3/10
Using the multiplication theorem: P(Ai β© W) = P(Ai) Γ P(W/Ai)
- P(A1 β© W) = 45/78 Γ 5/10 = 15/52
- P(A2 β© W) = 30/78 Γ 4/10 = 2/13
- P(A3 β© W) = 3/78 Γ 3/10 = 3/260
Finally, the addition theorem gives: P(W) = 15/52 + 2/13 + 3/260 = 59/130 = 0.45
Independent and Dependent Events
Two events A and B in the same sample space S are defined to be independent (or statistically independent) if the probability that one event occurs is not affected by whether the other event has or has not occurred.
π Definition β Independent Events: Events A and B are independent if and only if P(A/B) = P(A) and P(B/A) = P(B).
π Formula β Special Multiplication Theorem for Independent Events: P(A β© B) = P(A) Γ P(B) Plain-English meaning: For independent events, the probability that both occur is simply the product of their individual probabilities.
Rationale: The general multiplication theorem states P(A β© B) = P(A) Γ P(B/A). Substituting P(B/A) = P(B) yields the special case for independent events.
Two events A and B are defined to be dependent if P(A β© B) β P(A) Γ P(B). This means the occurrence of one event in some way affects the probability of the other event occurring.
π‘ Why this matters: Two events that are independent can never be mutually exclusive. Mutually exclusive events cannot occur simultaneously (P(Aβ©B)=0), while independent events require P(Aβ©B)=P(A)P(B), which can only be zero if at least one event has zero probability.
π Example β Dice Independence: Two fair dice, one red and one green, are thrown. Let A = red die shows an even number, and B = green die shows a 5 or 6.
- Sample space S has 36 outcomes
- P(A) = 3/6, P(B) = 2/6
- P(A β© B) = 6/36 (outcomes: (2,5), (4,5), (6,5), (2,6), (4,6), (6,6))
- P(A) Γ P(B) = 3/6 Γ 2/6 = 6/36 = P(A β© B)
- Therefore, events A and B are independent.
π Example β Birth Statistics Dependency: Using data from England and Wales (1956) on births by sex and whether liveborn or stillborn:
- P(stillbirth/male birth) = 8,609/368,490 = 0.0234
- P(stillbirth/female birth) = 7,796/348,258 = 0.0224
- Overall P(stillbirth) = 16,405/716,740 = 0.0229
Since the conditional probabilities differ from the unconditional probability, sex and stillbirth are statistically dependent. The sex of a baby has a small effect on its chance of being stillborn.
Marginal Probability
Marginal probabilities are the probabilities that appear in the margins of a contingency table.
Using the birth statistics data (Table-2):
| Liveborn | Stillborn | Total | |
|---|---|---|---|
| Male | 0.5021 | 0.0120 | 0.5141 |
| Female | 0.4750 | 0.0109 | 0.4859 |
| Total | 0.9771 | 0.0229 | 1.0000 |
π Definition β Marginal Probability: The probability of a single event (e.g., male birth, stillbirth) obtained by summing joint probabilities across the categories of another variable.
The marginal probability of male birth (0.5141) equals the sum of joint probabilities: P(male and liveborn) + P(male and stillborn) = 0.5021 + 0.0120 = 0.5141. This follows the addition theorem for mutually exclusive events.
The relationship between conditional, joint, and marginal probabilities is:
P(stillbirth/male birth) = P(male birth and stillbirth) Γ· P(male birth)
= 0.0120 Γ· 0.5141
= 0.0233
β Key Takeaways
The critical distinction between independent and dependent events lies in whether the occurrence of one event affects the probability of another. For independent events, the special multiplication theorem P(Aβ©B)=P(A)P(B) applies directly, while dependent events require conditional probabilities in the general multiplication theorem. Marginal probabilities, found in table margins, are obtained by summing joint probabilities across categories and represent unconditional probabilities of single events. The relationship P(A/B)=P(Aβ©B)/P(A) connects conditional, joint, and marginal probabilities. When analyzing real data, independence is rarely perfect, and slight deviations from independence (as shown in the birth statistics example) indicate relationships between variables.
π§ Quick Revision Questions
- What condition must be satisfied for two events A and B to be considered statistically independent?
- Why can independent events never be mutually exclusive?
- In the dice example, if P(A)=3/6, P(B)=2/6, and P(Aβ©B)=6/36, how do you verify these events are independent?
- How is a marginal probability computed from joint probabilities in a contingency table?
- Using the birth statistics data, explain why sex and stillbirth are considered dependent events despite the small difference in conditional probabilities.