π Lecture 28 β Division Algorithm
π Overview: This lecture focuses on proving the correctness of three fundamental algorithms using loop invariants: a loop to compute a product, the division algorithm for integer division with remainder, and the Euclidean algorithm for finding the greatest common divisor (gcd). Understanding these proofs is essential for establishing that algorithms reliably produce correct outputs for all valid inputs.
ποΈ Topics Covered
The lecture begins by proving the correctness of a simple loop that computes the product of a real number and a nonnegative integer using basis, inductive, and eventual falsity properties. It then proves the correctness of the division algorithm, which computes quotient and remainder for integer division. Finally, it introduces the Euclidean algorithm for computing the greatest common divisor of two integers, including a hand-calculation example, supporting lemma, and a formal proof of the algorithm's correctness using the loop invariant technique.
π Lecture Summary
CORRECTNESS OF: LOOP TO COMPUTE A PRODUCT
This section proves the correctness of a while loop that computes the product m Β· x where m is a nonnegative integer and x is a real number. The algorithm initializes i = 0 and product = 0, then repeatedly adds x to product and increments i until i = m.
The proof uses the loop invariant I(n): i = n and product = n Β· x.
I. Basis Property: I(0) holds before the first iteration because i = 0 and product = 0 Β· x = 0.
II. Inductive Property: Assuming I(k) is true (i.e., i = k and product = k Β· x) and the guard G: i β m is true, after one iteration we have product_new = product_old + x = kΒ·x + x = (k+1)Β·x and i_new = k + 1. Thus I(k+1) is true.
III. Eventual Falsity of Guard: The guard i β m becomes false after exactly m iterations because i starts at 0 and increments by 1 each time.
IV. Correctness of the Post-Condition: When the loop terminates after N = m iterations, I(m) is true, meaning i = m and product = m Β· x, exactly matching the post-condition.
THE DIVISION ALGORITHM
This section proves the correctness of an algorithm that computes integer division: given a nonnegative integer a and a positive integer d, it produces quotient q and remainder r such that a = qΒ·d + r and 0 β€ r < d. The algorithm initializes r = a and q = 0, then repeatedly subtracts d from r and increments q while r β₯ d.
The proof uses the loop invariant I(n): r = a - nΒ·d and n = q.
π Definition β Loop Invariant: A property that is true before the first iteration of a loop and remains true after each iteration.
I. Basis Property: I(0) is true because initially r = a = a - 0Β·d and q = 0.
II. Inductive Property: Assuming I(k): r = a - kΒ·d β₯ 0 and k = q and the guard r β₯ d is true, after one iteration: r_new = r - d = a - kΒ·d - d = a - (k+1)Β·d and q_new = k+1. Since r β₯ d, we have r_new β₯ 0, so I(k+1) is true.
III. Eventual Falsity of Guard: The guard r β₯ d becomes false after a finite number of iterations because r decreases by d each time and starts at a finite value a.
IV. Correctness of the Post-Condition: When the loop terminates, the guard is false (r < d) and I(N) is true (r = a - NΒ·d and N = q). Therefore a = qΒ·d + r and 0 β€ r < d, matching the post-condition.
π Formula: Division Algorithm: a = qΒ·d + r where 0 β€ r < d
π‘ Why this matters: The division algorithm is fundamental to number theory and computer arithmetic, forming the basis for many operations including modular arithmetic and the Euclidean algorithm.
THE EUCLIDEAN ALGORITHM
This section introduces the Euclidean algorithm, an efficient method for computing the greatest common divisor (gcd) of two integers. The gcd of two integers a and b is the largest integer that divides both.
π Definition β Greatest Common Divisor (gcd): The largest integer that divides both a and b. For example, gcd(12, 30) = 6.
Hand calculation example: Find gcd(330, 156)
The algorithm performs repeated division:
- Divide 330 by 156:
330 = 156Β·2 + 18 - Divide 156 by 18:
156 = 18Β·8 + 12 - Divide 18 by 12:
18 = 12Β·1 + 6 - Divide 12 by 6:
12 = 6Β·2 + 0
The last nonzero remainder is 6, so gcd(330, 156) = 6.
LEMMA: If a and b are any integers with b β 0 and q and r are nonnegative integers such that a = qΒ·b + r, then gcd(a, b) = gcd(b, r).
The formal algorithm initializes a = A, b = B, r = B (where A > B β₯ 0), then repeatedly replaces (a, b) with (b, a mod b) until b = 0. The loop invariant is I(n): gcd(a, b) = gcd(A, B) and 0 β€ b < a.
I. Basis Property: I(0) is true because gcd(a, b) = gcd(A, B) by initialization and 0 β€ b < a from the precondition.
II. Inductive Property: Assuming I(k) and the guard b β 0 are true, we have gcd(a_old, b_old) = gcd(A, B). After r_new = a_old mod b_old, we have a_old = b_oldΒ·q + r_new with 0 β€ r_new < b_old. By the lemma, gcd(a_old, b_old) = gcd(b_old, r_new) = gcd(A, B). After updating a_new = b_old and b_new = r_new, we get gcd(a_new, b_new) = gcd(A, B) and 0 β€ b_new < a_new, so I(k+1) is true.
III. Eventual Falsity of Guard: The guard b β 0 becomes false after a finite number of iterations because b is replaced by a mod b, which is always smaller than b and nonnegative.
IV. Correctness of the Post-Condition: When the loop terminates, b = 0 and I(N) is true: gcd(a, b) = gcd(A, B). Since gcd(a, 0) = a, we have a = gcd(A, B), matching the post-condition.
π Example: Using the Euclidean algorithm to find gcd(330, 156):
- Step 1: 330 Γ· 156 gives quotient 2, remainder 18 β
gcd(330, 156) = gcd(156, 18) - Step 2: 156 Γ· 18 gives quotient 8, remainder 12 β
gcd(156, 18) = gcd(18, 12) - Step 3: 18 Γ· 12 gives quotient 1, remainder 6 β
gcd(18, 12) = gcd(12, 6) - Step 4: 12 Γ· 6 gives quotient 2, remainder 0 β
gcd(12, 6) = gcd(6, 0) = 6
β Key Takeaways
The most critical concept from this lecture is the loop invariant technique for proving algorithm correctness, which requires establishing four properties: basis (invariant holds initially), induction (invariant preserved through iterations), eventual falsity of guard (loop terminates), and post-condition correctness. For the division algorithm, remember that the loop invariant r = a - nΒ·d along with the guard r β₯ d guarantees the post-condition a = qΒ·d + r with 0 β€ r < d. For the Euclidean algorithm, the key lemma gcd(a, b) = gcd(b, a mod b) allows repeated reduction of the problem until reaching gcd(d, 0) = d. The hand-calculation method using repeated division until a zero remainder is the practical implementation of this proof.
π§ Quick Revision Questions
- What are the four properties that must be proved to establish the correctness of a while loop using the loop invariant technique?
- In the division algorithm proof, what is the loop invariant and how does it lead to the post-condition
a = qΒ·d + rwith0 β€ r < d? - State the lemma that justifies the Euclidean algorithm's correctness: if
a = qΒ·b + r, then what is the relationship between gcd(a,b) and gcd(b,r)? - Using the Euclidean algorithm, find gcd(84, 30) showing all steps.
- In the Euclidean algorithm proof, why must the guard
b β 0eventually become false?
π Lecture 29 β Combinatorics
π Overview: This lecture introduces the fundamental principles of counting in combinatorics: the sum rule and product rule. These rules form the foundation for solving counting problems in computer science, probability, and algorithm analysis.
ποΈ Topics Covered
The lecture covers combinatorics as the mathematics of counting and arranging objects, the sum rule for counting mutually exclusive events, the product rule for counting sequential events, and applications including license plate combinations, bit strings, nested loop iterations, and variable naming conventions.
π Lecture Summary
COMBINATORICS
Combinatorics is the mathematics of counting and arranging objects. Counting objects with certain properties (enumeration) is required to solve many different types of problems.
For example, counting is used to:
- Determine number of ordered or unordered arrangement of objects
- Generate all the arrangements of a specified kind which is important in computer simulations
- Compute probabilities of events
- Analyze the chance of winning games, lotteries etc.
- Determine the complexity of algorithms
THE SUM RULE
If one event can occur in nβ ways, a second event can occur in nβ (different) ways, then the total number of ways in which exactly one of the events (i.e., first or second) can occur is nβ + nβ.
π Example: Suppose there are 7 different optional courses in Computer Science and 3 different optional courses in Mathematics. Then there are 7 + 3 = 10 choices for a student who wants to take one optional course.
π Example: A student can choose a computer project from one of three lists containing 23, 15 and 19 possible projects. The student can choose from the first list in 23 ways, from the second in 15 ways, and from the third in 19 ways. Hence, there are 23 + 15 + 19 = 57 projects to choose from.
π Generalized Sum Rule: If one event can occur in nβ ways, a second event can occur in nβ ways, a third event can occur in nβ ways, etc., then there are nβ + nβ + nβ + ... ways in which exactly one of the events can occur.
π Sum Rule in Terms of Sets: If Aβ, Aβ, ..., Aβ are finite disjoint sets, then the number of elements in the union of these sets is the sum of the number of elements in them. If n(Aα΅’) denotes the number of elements in set Aα΅’ for i = 1, 2, ..., m, then n(Aβ βͺ Aβ βͺ ... βͺ Aβ) = n(Aβ) + n(Aβ) + ... + n(Aβ) where Aα΅’ β© Aβ±Ό = β if i β j.
THE PRODUCT RULE
If one event can occur in nβ ways and if for each of these nβ ways, a second event can occur in nβ ways, then the total number of ways in which both events occur is nβ Β· nβ.
π Example: Suppose there are 7 different optional courses in Computer Science and 3 different optional courses in Mathematics. A student who wants to take one optional course of each subject has 7 Γ 3 = 21 choices.
π Example: The chairs of an auditorium are to be labeled with two characters, a letter followed by a digit. The procedure involves Assigning one of the 26 letters and Assigning one of the 10 digits. By product rule, there are 26 Γ 10 = 260 different ways a chair can be labeled.
π Generalized Product Rule: If some event can occur in nβ different ways, and following this event, a second event can occur in nβ different ways, and following this second event, a third event can occur in nβ different ways, etc., then the number of ways all events can occur in the order indicated is nβ Β· nβ Β· nβ Β· ...
π Product Rule in Terms of Sets: If Aβ, Aβ, ..., Aβ are finite sets, then the number of elements in the Cartesian product of these sets is the product of the number of elements in each set. n(Aβ Γ Aβ Γ ... Γ Aβ) = n(Aβ) Β· n(Aβ) Β· ... Β· n(Aβ)
π Example: An organization consisting of 15 members needs to elect a president, treasurer, and secretary (no person holds more than one position). The president can be elected in 15 ways; following this, the treasurer in 14 ways; following this, the secretary in 13 ways. Thus, n = 15 Γ 14 Γ 13 = 2730 different ways.
π Example: There are 4 bus lines between A and B, and 3 bus lines between B and C. (a) Travel from A to C via B: 4 Γ 3 = 12 ways (b) Round trip (AβBβCβBβA): 4 Γ 3 Γ 3 Γ 4 = 144 ways (c) Round trip without reusing a bus line: 4 Γ 3 Γ 2 Γ 3 = 72 ways (since only 2 ways from CβB and 3 ways from BβA remain)
π Example: A bit string is a sequence of 0's and 1's. For length 4, each bit can be chosen in 2 ways, so 2 Γ 2 Γ 2 Γ 2 = 2β΄ = 16 different bit strings.
π Example: Bit strings of length 8: (i) Begin with 1: First bit fixed (1 way), remaining 7 bits each 2 ways β 1 Γ 2β· = 128 (ii) Begin and end with 1: First and last fixed (1 way each), remaining 6 bits each 2 ways β 1 Γ 2βΆ Γ 1 = 64
π Example: License plates with three letters followed by three digits: (a) Total: 26 Γ 26 Γ 26 Γ 10 Γ 10 Γ 10 = 17,576,000 (b) Begin with A and end with 0: 1 Γ 26 Γ 26 Γ 10 Γ 10 Γ 1 = 67,600 (c) Begin with PQR: 1 Γ 1 Γ 1 Γ 10 Γ 10 Γ 10 = 1,000 (d) All letters and digits distinct: 26 Γ 25 Γ 24 Γ 10 Γ 9 Γ 8 = 11,232,000 (e) Begin with AB and all distinct: 1 Γ 1 Γ 24 Γ 10 Γ 9 Γ 8 = 17,280
π Example: Variable names are either a letter or a letter followed by a digit. Names of length 1: 26 ways. Names of length 2: 26 Γ 10 = 260 ways. By sum rule: 26 + 260 = 286 possible variable names.
π Example: Bit strings from one through four digits: 2 + 2Β² + 2Β³ + 2β΄ = 2 + 4 + 8 + 16 = 30. From five through eight digits: 2β΅ + 2βΆ + 2β· + 2βΈ = 480.
π Example: Three-digit integers divisible by 5 end in 0 or 5. Case I (end in 0): 9 choices for hundreds digit (1-9), 10 for tens digit (0-9), 1 for units digit = 9 Γ 10 Γ 1 = 90 Case II (end in 5): 9 Γ 10 Γ 1 = 90 By sum rule: 90 + 90 = 180 three-digit integers divisible by 5.
π Example: Access code of 1-3 letters (repetition allowed): 26ΒΉ + 26Β² + 26Β³ = 26 + 676 + 17,576 = 18,278.
NUMBER OF ITERATIONS OF A NESTED LOOP
π Example: Outer loop i = 1 to 4 (4 iterations), inner loop j = 1 to 3 (3 iterations per outer iteration). By product rule: 4 Γ 3 = 12 total inner loop iterations.
π Example: Outer loop i = 5 to 50 (50 - 5 + 1 = 46 iterations), inner loop j = 10 to 20 (20 - 10 + 1 = 11 iterations per outer iteration). By product rule: 46 Γ 11 = 506.
π Example: Outer loop i = 1 to 4, inner loop j = 1 to i. The inner loop iterates: 1, 2, 3, and 4 times respectively. By sum rule: 1 + 2 + 3 + 4 = 10 total iterations.
π‘ Why this matters: When inner loop bounds depend on the outer loop variable, the total iterations cannot be found by simple multiplication β we must sum the individual counts.
β Key Takeaways
The sum rule and product rule are the two fundamental counting principles in combinatorics. The sum rule applies when events are mutually exclusive (you choose exactly one), while the product rule applies when events occur in sequence (you choose one from each category). The sum rule generalizes to any finite number of disjoint sets where n(Aβ βͺ Aβ βͺ ... βͺ Aβ) = n(Aβ) + n(Aβ) + ... + n(Aβ). The product rule generalizes to any finite number of sequential events where n(Aβ Γ Aβ Γ ... Γ Aβ) = n(Aβ) Β· n(Aβ) Β· ... Β· n(Aβ). These rules combine to solve complex counting problems like license plate combinations, bit strings, and nested loop iterations, where the number of loop iterations can be computed using product rule for fixed bounds or sum rule for variable bounds.
π§ Quick Revision Questions
- How many different bit strings of length 6 are there?
- A restaurant offers 4 appetizers and 7 main courses. How many different meals consisting of one appetizer and one main course are possible?
- How many three-letter code words are possible using the English alphabet if letters can be repeated?
- For a nested loop where the outer loop runs from i = 1 to 5 and the inner loop runs from j = 1 to i, what is the total number of iterations of the inner loop?
- How many 4-digit numbers are there if the first digit cannot be 0?
π Lecture 32 β K-Combinations
π Overview: This lecture extends the concept of combinations to allow repetitions, introducing k-selections (also called k-combinations with repetition). It then covers ordered and unordered partitions of sets and permutations with repetitions (multinomial coefficients). These concepts are essential for solving complex counting problems where order may or may not matter, and where elements can be repeated or grouped.
ποΈ Topics Covered
This lecture begins with k-selections, defining them as unordered choices with repetition allowed, and presenting the formula C(k+n-1, k). It then provides a summary table comparing k-sample, k-permutation, k-combination, and k-selection. Next, it introduces ordered and unordered partitions of finite sets, with theorems and examples for dividing n elements into cells of specified sizes. Finally, it covers generalized permutations (permutations with repetitions), using multinomial coefficients to count arrangements of n elements where some are alike. Several exercises apply these concepts to real-world problems like distributing batteries, dividing students into groups, and arranging letters in words.
π Lecture Summary
K-SELECTIONS
k-selections are similar to k-combinations in that the order in which the elements are selected does not matter, but in this case repetitions can occur.
π Definition β k-selection: A k-selection of a set of n elements is a choice of k elements taken from a set of n elements such that the order of elements does not matter and elements can be repeated.
Remark: k-selections are also called k-combinations with repetition allowed or multisets of size k. With k-selections of a set of n elements, repetition of elements is allowed, so k need not to be less than or equal to n.
Theorem: The number of k-selections that can be selected from a set of n elements is C(k+n-1, k) or ( \binom{k+n-1}{k} ).
π Formula: ( \binom{k+n-1}{k} ) β This counts the number of ways to choose k items from n types when order doesn't matter and repetition is allowed.
π Example: A camera shop stocks ten different types of batteries. (a) How many ways can a total inventory of 30 batteries be distributed among the ten different types?
- Here k = 30, n = 10.
- Required number = C(30 + 10 β 1, 30) = C(39, 30) = ( \frac{39!}{(39-30)!30!} = 211,915,132 )
(b) Assuming one type is A76, how many ways can 30 batteries be distributed if the inventory must include at least four A76 batteries?
- Reserve 4 A76 batteries, leaving k = 26 batteries to distribute among n = 10 types.
- Required number = C(26 + 10 β 1, 26) = C(35, 26) = ( \frac{35!}{(35-26)!26!} = 70,607,460 )
WHICH FORMULA TO USE?
The lecture provides a summary table:
| ORDER MATTERS | ORDER DOES NOT MATTER | |
|---|---|---|
| REPETITION ALLOWED | k-sample: ( n^k ) | k-selection: C(n+k-1, k) |
| REPETITION NOT ALLOWED | k-permutation: P(n, k) | k-combination: C(n, k) |
ORDERED AND UNORDERED PARTITIONS
An unordered partition of a finite set S is a collection [Aβ, Aβ, ..., Aβ] of disjoint (nonempty) subsets of S (called cells) whose union is S. The partition is ordered if the order of the cells in the list counts.
π Example: Let S = {1, 2, 3, ..., 7}. The collections Pβ = [{1,2}, {3,4,5}, {6,7}] and Pβ = [{6,7}, {3,4,5}, {1,2}] determine the same partition of S but are distinct ordered partitions.
π Example: Suppose a box B contains seven marbles numbered 1 through 7. Find the number m of ways of drawing from B firstly two marbles, then three marbles and lastly the remaining two marbles.
- Order matters (the order of drawing is specified), so we count ordered partitions.
- m = C(7,2) Γ C(5,3) Γ C(2,2)
- = ( \frac{7!}{2!5!} \times \frac{5!}{2!3!} \times \frac{2!}{2!0!} )
- = ( \frac{7!}{2!3!2!} = 210 )
Theorem: Let S contain n elements and let nβ, nβ, ..., nβ be positive integers with nβ + nβ + ... + nβ = n. Then there exist ( \frac{n!}{n_1! n_2! n_3! \cdots n_k!} ) different ordered partitions of S of the form [Aβ, Aβ, ..., Aβ], where Aβ contains nβ elements, Aβ contains nβ elements, ..., Aβ contains nβ elements.
Remark: To find the number of unordered partitions, we have to count the ordered partitions and then divide it by a suitable number to erase the order in partitions.
π Example: Find the number m of ways that nine toys can be divided among four children if the youngest child is to receive three toys and each of the others two toys.
- This is ordered (children are distinct), with cells of sizes 3, 2, 2, 2.
- m = ( \frac{9!}{3!2!2!2!} = 2520 )
π Example: How many ways can 12 students be divided into 3 groups with 4 students in each group so that: (i) one group studies English, one History, and one Mathematics.
- Groups are labeled by subject, so ordered partitions.
- Number = ( \frac{12!}{4!4!4!} = 34,650 )
(ii) all the groups study Mathematics.
- Groups are unlabeled (same subject), so unordered partitions.
- Each unordered partition {Gβ, Gβ, Gβ} can be arranged in 3! ways as an ordered partition.
- Number = ( \frac{12!}{4!4!4!} \times \frac{1}{3!} )
π Example: How many ways can 8 students be divided into two teams containing: (i) five and three students respectively.
- Teams have different sizes, so each unordered partition gives only one ordered partition.
- Number = ( \frac{8!}{5!3!} = 56 )
(ii) four students each.
- Teams are unlabeled and equal size.
- Ordered partitions = ( \frac{8!}{4!4!} = 70 ). Since each unordered partition determines 2! = 2 ordered partitions, number = 70/2 = 35.
π Example: Find the number m of ways that a class X with ten students can be partitioned into four teams Aβ, Aβ, Bβ and Bβ where Aβ and Aβ contain two students each and Bβ and Bβ contain three students each.
- Ordered partitions = ( \frac{10!}{2!2!3!3!} = 25,200 )
- Each unordered partition [Aβ, Aβ, Bβ, Bβ] determines 2!Β·2! = 4 ordered partitions.
- m = 25,200 / 4 = 6,300
π Example: Suppose 20 people are divided into 6 (numbered) committees so that 3 people each serve on Cβ and Cβ, 4 people each on Cβ and Cβ, 2 people on Cβ , and 4 people on Cβ. How many possible arrangements?
- Committees are labeled, so ordered partitions.
- Number = ( \frac{20!}{3!3!4!4!2!4!} = 2,444,321,880,000 )
π Example: If 20 people are divided into teams of size 3, 3, 4, 4, 2, 4, find the number of possible arrangements.
- Now teams (cells) are unlabeled.
- Since we have two teams of size 3 (swap gives 2! ways) and three teams of size 4 (swap gives 3! ways), we divide by 3!2! to remove order.
- Number = ( \frac{20!}{3!3!4!4!2!4!} \times \frac{1}{3!2!} = 203,693,490,000 )
GENERALIZED PERMUTATION or PERMUTATIONS WITH REPETITIONS
The number of permutations of n elements of which nβ are alike, nβ are alike, ..., nβ are alike is ( \frac{n!}{n_1! n_2! \cdots n_k!} ).
Remark: The number ( \frac{n!}{n_1! n_2! \cdots n_k!} ) is often called a multinomial coefficient, and is denoted by the symbol ( \binom{n}{n_1, n_2, \cdots, n_k} ).
π Example: Find the number of distinct permutations that can be formed using the letters of the word "BENZENE".
- 7 letters: three E's (alike) and two N's (alike).
- Number = ( \frac{7!}{3!2!} = 420 )
π Example: How many different signals each consisting of six flags hung in a vertical line can be formed from four identical red flags and two identical blue flags?
- 6 elements: 4 alike and 2 alike.
- Number = ( \frac{6!}{4!2!} = 15 )
π Example: (i) Find the number of "words" that can be formed from the letters of the word ELEVEN.
- 6 letters: three E's.
- Number = ( \frac{6!}{3!} = 120 )
(ii) Find, if the words are to begin with L.
- First letter fixed as L, remaining 5 letters with three E's.
- Number = ( \frac{5!}{3!} = 20 )
(iii) Find, if the words are to begin and end in E.
- First and last fixed as E, remaining 4 distinct letters.
- Number = 4! = 24
(iv) Find, if the words are to begin with E and end in N.
- First fixed as E, last fixed as N, remaining 4 letters with two E's.
- Number = ( \frac{4!}{2!} = 12 )
π Example: (i) Find the number of permutations from all the letters of the word BASEBALL.
- 8 letters: two B's, two A's, two L's.
- Number = ( \frac{8!}{2!2!2!} = 5,040 )
(ii) Find, if the two B's are to be next to each other.
- Treat two B's as one letter: 7 letters with two A's and two L's.
- Number = ( \frac{7!}{2!2!} = 1,260 )
(iii) Find, if the words are to begin and end in a vowel.
- Vowels are A and E. Three cases: both A, A then E, E then A (3 possibilities).
- For each case, 6 positions left with two B's and two L's.
- Number = ( 3 \times \frac{6!}{2!2!} = 3 \times 180 = 540 )
β Key Takeaways
- A k-selection counts unordered selections with repetition allowed, using the formula ( \binom{k+n-1}{k} ). This is distinct from k-combinations (no repetition), k-permutations (ordered, no repetition), and k-samples (ordered, repetition allowed). For ordered partitions of n elements into cells of specified sizes, use the multinomial coefficient ( \frac{n!}{n_1! n_2! \cdots n_k!} ). For unordered partitions, you must divide by the factorial of the number of cells of the same size to account for order.
- Permutations with repetitions (generalized permutations) also use the multinomial coefficient formula to count distinct arrangements when some elements are identical. Memorize the formula ( \frac{n!}{n_1! n_2! \cdots n_k!} ) for both ordered partitions and permutations with repetitions, and be able to distinguish between ordered and unordered partitions based on whether the groups/cells are labeled.
π§ Quick Revision Questions
- What is the formula for the number of k-selections from a set of n elements, and what does it mean?
- When dividing 10 students into two unlabeled groups of 5 each, why is the number of ways NOT ( \frac{10!}{5!5!} )?
- How many distinct permutations can be formed from the letters of the word "MISSISSIPPI"?
- In the battery inventory problem, why is the number of ways to distribute 30 batteries among 10 types given by C(30+10-1, 30)?
- What is the difference between an ordered partition and an unordered partition of a set?
π Lecture 33 β Tree Diagram
π Overview: This lecture introduces the tree diagram as a systematic tool for listing all logical possibilities in a sequence of events where each event has a finite number of outcomes. It then presents the inclusion-exclusion principle for counting elements in unions of sets, a fundamental counting technique used in probability and combinatorics.
ποΈ Topics Covered
The lecture begins with tree diagrams, explaining their structure and demonstrating their use with examples such as permutations, product sets, tournament outcomes, bit strings without consecutive 1s, and officer selection with constraints. It then shifts to the inclusion-exclusion principle, starting with the basic formula for two sets, followed by multiple exercises applying the principle to problems involving overlapping sets, complements, and differences.
π Lecture Summary
TREE DIAGRAM
A tree diagram is a useful tool to list all the logical possibilities of a sequence of events where each event can occur in a finite number of ways. A tree consists of a root, a number of branches leaving the root, and possible additional branches leaving the end points of other branches. To use trees in counting problems, we use a branch to represent each possible choice. The possible outcomes are represented by the leaves (end points of branches). A tree is normally constructed from left to right.
π Definition β Tree Diagram: A diagram consisting of a root, branches, and leaves used to enumerate all possible outcomes of a sequence of events.
π Example: Find the permutations of {a, b, c}. The number of permutations of 3 elements is P(3, 3) = 3!/(3-3)! = 3! = 6. The tree diagram shows branches from the root for a, b, c. From a, branches go to b and c; from b, branches go to a and c; from c, branches go to a and b. This yields the six permutations: abc, acb, bac, bca, cab, cba.
π Example: Find the product set A Γ B Γ C, where A = {1,2}, B = {a,b,c}, and C = {3,4} by constructing the appropriate tree diagram. The tree starts from the root with branches for 1 and 2. From 1, branches go to a, b, c. From each of these, branches go to 3 and 4. Similarly for 2. This yields 12 elements: (1,a,3), (1,a,4), (1,b,3), (1,b,4), (1,c,3), (1,c,4), (2,a,3), (2,a,4), (2,b,3), (2,b,4), (2,c,3), (2,c,4).
π Example: Teams A and B play in a tournament. The team that first wins two games wins the tournament. Find the number of possible ways the tournament can occur. The tree shows the first game results in A or B. From A, the second game: if A wins again (AA β tournament over) or B wins (AB β third game needed). Similarly from B. The tournament can occur in 6 ways: AA, ABA, ABB, BAA, BAB, BB.
π Example: How many bit strings of length four do not have two consecutive 1βs? The tree diagram displays all bit strings of length four without two consecutive 1βs. The first bit can be 0 or 1. If first bit is 0, second can be 0 or 1. If first bit is 1, second can only be 0. Continuing this pattern yields 8 bit strings: 0000, 0001, 0010, 0100, 0101, 1000, 1001, 1010.
π Example: Three officers β a president, a treasurer, and a secretary β are to be chosen from among four possible: A, B, C, and D. Suppose that A cannot be president and either C or D must be secretary. How many ways can the officers be chosen? The possibility tree starts with the president selection. Since A cannot be president, the president can be B, C, or D. From each president, branches show possible treasurers (remaining people). From each treasurer, branches show possible secretaries (either C or D must be chosen). There are only eight ways possible to choose the offices under given conditions.
THE INCLUSION-EXCLUSION PRINCIPLE
- If A and B are disjoint finite sets, then n(A βͺ B) = n(A) + n(B).
- If A and B are finite sets (not necessarily disjoint), then n(A βͺ B) = n(A) + n(B) - n(A β© B).
π‘ Why this matters: Statement 1 follows from the sum rule. Statement 2 corrects for double-counting: elements in A β© B are counted twice (once in n(A) and once in n(B)), so we subtract n(A β© B) once to get the correct count for n(A βͺ B).
π Definition β Inclusion-Exclusion Principle (Two Sets): For any finite sets A and B, n(A βͺ B) = n(A) + n(B) - n(A β© B).
π Example: There are 15 girl students and 25 boy students in a class. How many students are there in total? Since the sets of boy and girl students are disjoint, n(G βͺ B) = n(G) + n(B) = 15 + 25 = 40.
π Example: Among 200 people, 150 either swim or jog or both. If 85 swim and 60 swim and jog, how many jog? n(S βͺ J) = n(S) + n(J) - n(S β© J) β 150 = 85 + n(J) - 60 β n(J) = 150 - 85 + 60 = 125. Hence 125 people jog.
π Example: Suppose A and B are finite sets. Show that n(A\B) = n(A) - n(A β© B). Set A may be written as the union of two disjoint sets A\B and A β© B. Hence, by the inclusion-exclusion principle (for disjoint sets): n(A) = n(A\B) + n(A β© B) β n(A\B) = n(A) - n(A β© B).
π Definition β Complement: n(A') = n(U\A) = n(U) - n(U β© A) = n(U) - n(A).
π Example: Let A and B be subsets of U with n(A) = 10, n(B) = 15, n(A') = 12, and n(A β© B) = 8. Find n(A βͺ B'). n(A βͺ B') = n(U \ (B \ A)) = n(U) - n(B\A). First, n(U) = n(A) + n(A') = 10 + 12 = 22. Then, n(B\A) = n(B) - n(A β© B) = 15 - 8 = 7. Therefore, n(A βͺ B') = 22 - 7 = 15.
π Example: Let A and B be subsets of U with n(U) = 100, n(A) = 50, n(B) = 60, and n((A βͺ B)') = 20. Find n(A β© B). n((A βͺ B)') = n(U) - n(A βͺ B) β 20 = 100 - n(A βͺ B) β n(A βͺ B) = 80. By inclusion-exclusion: n(A βͺ B) = n(A) + n(B) - n(A β© B) β 80 = 50 + 60 - n(A β© B) β n(A β© B) = 50 + 60 - 80 = 30.
π Example: Suppose 18 people read English newspaper (E) or Urdu newspaper (U) or both. Given 5 people read only English newspaper and 7 read both, find the number βrβ of people who read only Urdu newspaper. n(E βͺ U) = n(E\U) + n(E β© U) + n(U\E) β 18 = 5 + 7 + r β r = 18 - 5 - 7 = 6.
π Example: Fifty people are interviewed. 20 like Chinese food (C), 32 like fast food (F), and 12 like neither. How many like Chinese but not fast food? n(C βͺ F) = n(C\F) + n(F). First, n((C βͺ F)') = n(U) - n(C βͺ F) β 12 = 50 - n(C βͺ F) β n(C βͺ F) = 38. Then, 38 = n(C\F) + 32 β n(C\F) = 38 - 32 = 6.
β Key Takeaways
Tree diagrams are essential for visual enumeration of all possible outcomes in sequential events. They are especially useful when constraints limit certain branches. The inclusion-exclusion principle is fundamental: for two sets, always subtract the intersection to avoid double-counting when sets overlap. The relationship between set difference and intersection (n(A\B) = n(A) - n(A β© B)) is a key formula derived from the principle. When working with complements, always first find the total n(U) from the union of a set and its complement. For problems involving "only" one category, break the union into its three disjoint parts: only A, both A and B, and only B.
π§ Quick Revision Questions
- In constructing a tree diagram for finding bit strings of length four without two consecutive 1's, what restriction applies to the second bit if the first bit is 1?
- What is the inclusion-exclusion formula for n(A βͺ B) when A and B are finite sets?
- If n(U) = 50, n(A) = 25, n(B) = 30, and n(A' β© B') = 10, what is n(A βͺ B)?
- How do you express n(A\B) in terms of n(A) and n(A β© B)?
- In the officer selection example, why were there only 8 possible ways even though there were 4 candidates?
π Lecture 34 β Inclusion-Exclusion Principle
π Overview: This lecture introduces the inclusion-exclusion principle for counting elements in unions of sets, extending it to three and four sets. It also introduces the pigeonhole principle, a fundamental counting argument used to guarantee the existence of repetitions or patterns. These principles are essential for solving combinatorial problems in probability, computer science, and everyday reasoning.
ποΈ Topics Covered
The lecture covers the inclusion-exclusion principle for two, three, and four sets with worked examples, a revision of functions, the pigeonhole principle with its generalized form, and the definitions of floor and ceiling functions. Several exercises demonstrate how to apply both principles to find minimum numbers needed to guarantee a certain outcome.
π Lecture Summary
Inclusion-Exclusion Principle
The inclusion-exclusion principle is a counting technique used to find the number of elements in the union of two or more sets. For two sets A and B, it accounts for elements counted twice (the intersection) by subtracting it once. The principle can be extended to three and four sets.
π Definition β Inclusion-Exclusion Principle for Two Sets: For finite sets A and B, the number of elements in the union of A and B is given by: n(A βͺ B) = n(A) + n(B) - n(A β© B).
π Formula: n(A βͺ B) = n(A) + n(B) - n(A β© B) β This formula adds the elements in each set and subtracts the elements counted twice (those in both sets).
π Example: How many integers from 1 through 1000 are multiples of 3 or 5? Let A = multiples of 3, B = multiples of 5. We compute:
n(A) = β1000/3β = 333n(B) = β1000/5β = 200n(A β© B) = β1000/15β = 66(multiples of both 3 and 5, i.e., multiples of 15)- Therefore,
n(A βͺ B) = 333 + 200 - 66 = 467.
Example (continued): How many integers from 1 through 1000 are neither multiples of 3 nor 5? This is the complement of A βͺ B. Since the universal set U has 1000 elements: n((A βͺ B)') = n(U) - n(A βͺ B) = 1000 - 467 = 533.
π‘ Why this matters: The complement approach (neither...nor) is a common application of the inclusion-exclusion principle.
Inclusion-Exclusion Principle for 3 and 4 Sets
The principle generalizes to more than two sets, alternating between adding and subtracting the sizes of intersections of increasing numbers of sets.
π Formula for Three Sets: n(A βͺ B βͺ C) = n(A) + n(B) + n(C) - n(A β© B) - n(B β© C) - n(A β© C) + n(A β© B β© C).
π Formula for Four Sets: n(A βͺ B βͺ C βͺ D) = n(A) + n(B) + n(C) + n(D) - n(A β© B) - n(A β© C) - n(A β© D) - n(B β© C) - n(B β© D) - n(C β© D) + n(A β© B β© C) + n(A β© B β© D) + n(A β© C β© D) + n(B β© C β© D) - n(A β© B β© C β© D).
π Example: A survey of 100 college students: 8 owned a car (C), 20 a motorcycle (M), 48 a bicycle (B), 38 owned none of these, and no student owned both a car and a motorcycle (n(C β© M) = 0). How many students owned a bicycle and either a car or a motorcycle (i.e., n(B β© C) + n(B β© M))?
- First, find
n(C βͺ M βͺ B) = n(U) - n((C βͺ M βͺ B)') = 100 - 38 = 62. - Apply the three-set formula:
62 = 8 + 20 + 48 - 0 - n(C β© B) - n(M β© B) + n(C β© M β© B). - Given no student has a car and a motorcycle, the triple intersection
n(C β© M β© B) = 0. - Therefore:
62 = 76 - n(C β© B) - n(M β© B), son(C β© B) + n(M β© B) = 76 - 62 = 14. Thus, 14 students owned a bicycle and either a car or a motorcycle.
Revision of Functions
A function from set A to set B is a relation where each element of A has exactly one image in B. A relation is not a function if any element of the domain has no image or more than one image.
Pigeonhole Principle
The pigeonhole principle is a simple but powerful counting argument. The classic formulation involves pigeons and holes.
π Definition β Pigeonhole Principle: A function from a set of k + 1 or more elements to a set of k elements must have at least two elements in the domain that have the same image in the co-domain. If k+1 or more pigeons fly into k pigeonholes, at least one pigeonhole contains two or more pigeons.
π Examples:
- Among any 367 people, at least two have the same birthday (only 366 possible birthdays).
- In any set of 27 English words, at least two begin with the same letter (26 letters).
π Example: What is the minimum number of students to guarantee two are born in the same month? There are 12 months (pigeonholes). By the principle, any 13 (= 12 + 1) or more students guarantees at least two in the same month.
π Example: Given any 7 integers, must two have the same remainder when divided by 6? Possible remainders are {0, 1, 2, 3, 4, 5} (6 pigeonholes). With 7 (= 6 + 1) integers, by the pigeonhole principle, at least two will have the same remainder.
π Example: How many integers from 1 to 100 must be picked to guarantee one divisible by 5? There are 20 integers divisible by 5 and 80 that are not. To be sure of getting a multiple of 5, you must pick 80 + 1 = 81 integers (pigeonhole principle).
π Example: Let A = {1, 2, ..., 10}. Choose 6 integers from A. Must two have a sum of 11? Partition A into 5 subsets (pigeonholes): {1,10}, {2,9}, {3,8}, {4,7}, {5,6}, each summing to 11. Choosing 6 (= 5 + 1) integers guarantees two from the same subset, whose sum is 11.
Generalized Pigeonhole Principle
The principle can be extended to guarantee more than two elements sharing a property.
π Definition β Generalized Pigeonhole Principle: A function from a set of nΒ·k + 1 or more elements to a set of n elements must have at least k + 1 elements in the domain that have the same image in the co-domain. If nΒ·k+1 or more pigeons fly into n pigeonholes, at least one pigeonhole contains k+1 or more pigeons.
π Example: A laundry bag has red, white, and blue socks. Find the minimum number to choose to guarantee 2 pairs (4 socks) of the same colour. Here, n = 3 colours (pigeonholes), and we want k+1 = 4, so k = 3. Thus, the minimum is nΒ·k + 1 = 3Β·3 + 1 = 10 socks.
Floor and Ceiling Functions
These functions round a real number to the nearest integer.
π Definition β Floor of x: Denoted βxβ, it is the largest integer smaller than or equal to x.
π Definition β Ceiling of x: Denoted βxβ, it is the smallest integer greater than or equal to x.
π Example: Compute floor and ceiling for:
- 25/4 = 6.25:
β25/4β = 6,β25/4β = 7 - 0.999:
β0.999β = 0,β0.999β = 1 - -2.01:
β-2.01β = -3,β-2.01β = -2(since -3 < -2.01 and -2 > -2.01).
π Example: Find the smallest integer N such that βN/7β = 5. The smallest N is N = 7Β·(5 - 1) + 1 = 7Β·4 + 1 = 29.
π Example: Find the smallest integer N such that βN/9β = 6. The smallest N is N = 9Β·(6 - 1) + 1 = 9Β·5 + 1 = 46.
Alternative Form of the Pigeonhole Principle
This form uses the ceiling function.
π Definition β Ceiling Form of the Pigeonhole Principle: If N pigeons fly into k pigeonholes, then at least one pigeonhole must contain βN/kβ or more pigeons.
π Example: Among 100 people, at least β100/12β = β8.333...β = 9 were born in the same month.
π Example: Minimum number of students in a class to guarantee at least 6 receive the same grade (A, B, C, D, F = 5 grades)? We need the smallest N such that βN/5β = 6. The smallest N is N = 5(6-1) + 1 = 5Β·5 + 1 = 26.
β Key Takeaways
The inclusion-exclusion principle is a cornerstone of counting that effectively handles overlapping sets by systematically adding and subtracting intersections of increasing size; for two sets it's n(AβͺB) = n(A) + n(B) - n(Aβ©B). The pigeonhole principle, in its various forms, is a powerful tool for proving the existence of repetitions or guaranteed outcomes: k+1 items placed into k categories guarantees at least one category with at least two items. The generalized pigeonhole principle (using nΒ·k+1 items for n categories to guarantee k+1 in one category) and its ceiling form (βN/kβ gives the minimum guaranteed in one hole) extend its applicability to more complex problems. Finally, ceiling and floor functions are used to precisely express the minimum number of items needed to guarantee a desired count in the generalized principle.
π§ Quick Revision Questions
- How many elements in the union of three sets
A,B, andCifn(A)=5,n(B)=6,n(C)=7,n(Aβ©B)=2,n(Bβ©C)=3,n(Aβ©C)=1, andn(Aβ©Bβ©C)=1? - If you have 10 pigeons and 4 pigeonholes, what is the minimum number of pigeons guaranteed in at least one hole, according to the ceiling form of the pigeonhole principle?
- State the generalized pigeonhole principle in your own words.
- What is the smallest positive integer N such that
βN/3β = 10? - Out of 30 students in a class, what is the minimum number that must share the same birth month, and which form of the pigeonhole principle allows you to determine this?