π Lecture 42 β Collision & Linear Probing
π Overview: This lecture addresses the fundamental problem of collision in hashing β when two different keys produce the same hash index. It introduces three collision resolution strategies, with detailed focus on linear probing, including its implementation for insert, find, and delete operations, and the clustering problem that arises. Understanding collision resolution is critical for implementing efficient hash tables in real-world applications.
ποΈ Topics Covered
The lecture begins by discussing hash functions for integer keys and the importance of using prime table sizes to avoid systematic collisions. It formally defines collision and presents three primary solutions: searching for an empty location (linear probing), using multiple hash functions, and using linked lists at each array position. The bulk of the lecture then details linear probing, including how insertion, lookup, and deletion work with wrap-around, the need for three cell states (occupied, empty, deleted), and the clustering problem that degrades performance. Quadratic probing is briefly introduced as an improvement.
π Lecture Summary
Hash Function for Integer Keys & Prime Table Size
If keys are integers, a good hash function is key % T where T is the table size. However, if T=10 and all keys end in 0, then key % 10 = 0 for all keys β a catastrophic collision scenario. To avoid such problems, T should be a prime number. When storing 100 records, choose a prime number near 100 as MAXTABLESIZE. Using a prime number helps resolve problems where keys share common factors with the table size, though it cannot guarantee unique hash values for all keys.
π Definition β hash function for integers: key % T where T should be a prime number to minimize systematic collisions.
π Formula: hash(key) = key % TableSize β takes the remainder when the key is divided by the table size to produce an array index.
π Example: If T=10 and all employee IDs end in 0 (e.g., 100, 200, 300), then hash(100) = 0, hash(200) = 0, hash(300) = 0 β all keys map to index 0. Using a prime T like 11 would give different values: 100 % 11 = 1, 200 % 11 = 2, 300 % 11 = 3.
Collision
Collision occurs when two or more keys (data items) produce the same hash index. This is inevitable and not the hash function's fault β the hash function is a mathematical formula that takes keys and returns a number. The caller (our ADT implementation) is responsible for handling collisions using a collision resolution strategy. The fundamental principle is "first come, first served" β the first value that hashes to a location gets it; subsequent values must be handled differently.
π Definition β collision: When two values hash to the same array location.
π Example: In a fruit table, hash("mango") = 6 and mango is stored at position 6. When inserting hash("honeydew") = 6, a collision occurs because position 6 is already occupied. The resolver must find another location for honeydew.
Three collision resolution solutions are presented:
Solution #1: Search for an empty location β Stop searching when we find the value or an empty location. Search must wrap-around at the end.
Solution #2: Use a second hash function β Then a third, fourth, fifth, etc. Call these hash functions one by one until an empty location is found.
Solution #3: Use the array location as the header of a linked list β The array becomes an array of pointers to TableNode. Each list node stores the data. When collision occurs, insert the new node at the front of the list at that position.
π‘ Why this matters: Collision resolution is not optional β it is guaranteed to happen with real-world data. Choosing the right strategy affects both time efficiency and memory usage.
Linear Probing
Linear probing is the first solution (open addressing/closed hashing). When a collision occurs, the array is scanned sequentially (with wrap-around) in search of an empty cell. More formally, cells at hβ(x), hβ(x), hβ(x), ... are tried in succession where:
hi(x) = (hash(x) + f(i)) mod TableSize, with f(0) = 0
For linear probing, f(i) = i (a linear function), so:
location(x) = (hash(x) + i) mod TableSize
π Definition β linear probing: A collision resolution strategy that scans the array sequentially (with wrap-around) in search of an empty cell by adding 1, 2, 3, etc. to the original hash value.
π Formula: location(x) = (hash(x) + i) mod TableSize where i starts at 0 and increments by 1 each time a collision occurs.
π Example β Insertion with collisions: Given a partial array with birds:
- Position 142: robin
- Position 143: sparrow
- Position 144: hawk
- Position 145: empty
- Position 147: bluejay
- Position 148: owl
Inserting "seagull" where hashCode("seagull") = 143:
- Check position 143: occupied by sparrow (not empty, not equal to seagull)
- Add 1 β position 144: occupied by hawk (not empty, not equal)
- Add 2 β position 145: empty β store seagull here (2 collisions resolved)
π Example β Find operation:
To find "hawk" where hashCode("hawk") = 143:
- Check position 143: occupied by sparrow (not hawk)
- Add 1 β position 144: occupied by hawk β found (1 probe after initial check)
π Example β Wrap-around:
To insert "cardinal" where hashCode("cardinal") = 147 and position 147 is occupied by bluejay, position 148 is occupied by owl, and 148 is the last position:
- Check 147: occupied by bluejay
- Add 1 β position 148: occupied by owl
- Wrap around to position 0: check, then 1, 2, etc. until empty space found
Three states for deletion: When using linear probing, deletion creates a "hole" problem. If an element in the middle of a chain is deleted, the probe may incorrectly stop at the empty cell and conclude the target item is not in the table. Solution: maintain three states for each cell:
- Occupied: has valid data
- Empty (never used): no data has ever been stored here
- Deleted (previously used): had data that has been removed
Clustering problem: Linear probing tends to form clusters β groups of items not containing any open slots. The bigger a cluster gets, the more likely new values will hash into it, making it even bigger. Clusters cause efficiency to degrade from O(1) to near O(n).
π Example of clustering: With three consecutive collisions at positions 143, 144, and 145 (sparrow, hawk, seagull), the next item hashing to 143 would be placed at 146, extending the cluster further. This behaves almost like sequential array storage instead of scattered hashing.
Quadratic probing is introduced as a solution to clustering:
- Use
F(i) = iΒ²(square of i) to resolve collisions - If hash function resolves to H and cell H is occupied, try H+1Β², H+2Β², H+3Β², ...
π Formula: location(x) = (hash(x) + iΒ²) mod TableSize
π Example: Inserting seagull with quadratic probing:
hash("seagull") = 143, position 143 occupied- Add 1Β² β position 144: occupied
- Add 2Β² β position 147: if available, store here (data now scattered instead of clustered)
Linked list approach comparison:
- Advantages over open addressing: simpler insertion and removal, array size is not a limitation
- Disadvantage: memory overhead is large if entries are small
The array becomes an array of pointers, and each position has a linked list. When collision occurs, the new item is inserted at the front of the linked list at that position.
β Key Takeaways
Collision is inevitable in hashing and must be handled by the ADT implementation, not the hash function. Linear probing is the simplest approach β it scans sequentially for empty cells using (hash(x) + i) mod TableSize β but suffers from clustering, which degrades performance from O(1) toward O(n). To handle deletions correctly, cells must have three states (occupied, empty, deleted) so that probe chains are not broken. Quadratic probing (using iΒ² instead of i) helps scatter data to reduce clustering, while linked list chaining eliminates the array-full problem at the cost of memory overhead. The fundamental trade-off is between simplicity and efficiency β no single collision resolution strategy is perfect for all scenarios.
π§ Quick Revision Questions
- Why should table size be a prime number when using the
key % Thash function for integer keys? - What is the formula for linear probing and how does wrap-around work when probing reaches the end of the array?
- Explain the clustering problem in linear probing β why does it occur and how does it affect performance?
- Why are three cell states (occupied, empty, deleted) needed for deletion in linear probing? What would happen with only two states?
- How does quadratic probing differ from linear probing, and what advantage does it offer over linear probing?
π Lecture 43 β Data Structures
π Overview: This lecture concludes the discussion of hashing with animations of collision resolution strategies and explores practical applications where hashing is suitable or unsuitable. It then introduces the topic of sorting, beginning with elementary algorithms like selection sort, which serve as baseline for comparing more efficient sorting methods.
ποΈ Topics Covered
The lecture covers hashing animation demonstrating linear probing, quadratic probing, and linked list chaining in a Java applet. It then discusses practical applications of hashing including compiler symbol tables, spell checkers, game programming, and inequality checking. The lecture examines when hashing is suitable versus when other data structures like AVL trees are better. Finally, it introduces sorting concepts and elementary sorting algorithms, focusing on selection sort.
π Lecture Summary
Hashing Animation
In the previous lecture, we discussed collision strategies in hashing including Linear Probing, Quadratic Probing, and Linked List chaining. Hashing is a vast research field covering hash functions, storage, and collision issues. Using hashing, operations of insert, delete, and find are performed in constant time, meaning the time does not increase with data volume. However, when collisions occur, the time does not remain constant. With linear probing, data must be inserted by sorting the array sequentially. With quadratic probing, a similar sequential approach is used. With linked list chaining, constructing linked lists takes time and memory.
The lecture demonstrates these three strategies using animations in a Java applet. The applet shows an array of size 100 with indices from 0 to 99. Each array element has two locations, so 200 elements can be stored. When the first collision occurs, the program uses the 2nd part of the array location. When a 2nd collision occurs, data is stored using the chosen collision resolution method.
The hash function used is x mod 100, meaning when a number is passed to it, it takes mod with 100 and returns the result used as the array index. The program generates 100 random numbers and stores them using the hash function.
π Definition β Hashing: A technique that maps data of arbitrary size to fixed-size values (hash values) using a hash function, enabling constant-time operations for insert, delete, and find.
π Formula: hash value = key mod tableSize β The remainder when dividing the key by the table size determines the array index for storing or retrieving data.
π Example: Number 506 is divided by 100, remainder is 6, so it is stored at array location 6. Number 206 also has remainder 6, so it is stored in the 2nd part of location 6. Number 806 has remainder 6 but both parts of location 6 are occupied, so using linear probing it is stored at location 7. Number 807 has remainder 7, but location 7 is already occupied due to linear probing, so it is stored at location 8. This demonstrates a clustering effect where numbers with remainder 63 cluster around location 63.
For quadratic probing, the array size is 75 and each location stores two numbers. In quadratic probing, we add the square of 1 (1), then square of 2 (4), then square of 3 (9), and so on in case of collisions. The hash function uses mod with 75.
For linked list chaining, the hash function uses 50 to take mod with numbers. When both parts of a location are filled, a linked list appears attached to that location. For example, four numbers have remainder 0 β two are stored in the array and the next two are stored using the link list attached at location 0.
The lecture notes that hashing is not covered in great depth here as it belongs to algorithms and analysis of algorithms domain, which is not part of this course.
Applications of Hashing
Compilers use hash tables to keep track of declared variables (symbol table). A symbol table is an important part of the compilation process. The compiler puts variables inside the symbol table during compilation, keeping track of different attributes including variable name, type, scope, and function name where declared. Operations on the symbol table include insertion of variable information, search for variable information, or deletion of a variable. Insert and find are the most commonly used operations. A variable name serves as the parameter (or key) to these operations. However, if a variable named x exists outside a code block and another variable with the same name and type is declared inside that block, scope becomes the differentiating factor. Assuming all variables have unique names, the variable name can be used as the key. The compiler inserts variable information by calling the insert function and retrieves variable values by passing the variable name.
A hash table can be used for on-line spelling checkers β if misspelling detection (rather than correction) is important, an entire dictionary can be hashed and words checked in constant time. To find spelling mistakes, first take all words from the dictionary and construct a hash table. To check a text, take each word and compare it with all words in the hash table. If a word is not found, there is a high probability it is incorrect, though there is a low probability the word is correct but absent from the dictionary. Based on this high probability, a message can be displayed to the user.
Game playing programs use hash tables to store seen positions, thereby saving computation time if the position is encountered again. Consider chess where positions of pieces (64 pieces) can be used as the key to store in the hash table. If the program wants to analyze whether a player has encountered a similar situation before, it passes the positions to the find function. When the positions are hashed again, the previously present index is returned, indicating the situation was encountered before.
Hash functions can be used to quickly check for inequality β if two elements hash to different values, they must be different. Sometimes you only need to know if values are equal or not, not which is smaller or larger. If two data items don't collide, their hash values will be different, indicating the values are unequal.
π‘ Why this matters: Hashing enables constant-time operations for many real-world applications, making it invaluable for compiler design, spell checking, game AI, and equality testing β but understanding its limitations is equally important for choosing the right data structure.
When Hashing is Suitable?
Hash tables are very good if there is a need for many searches in a reasonably stable table. The dictionary hash table example demonstrates this β the hash table was constructed once and lookup operations were frequent while insertions occurred very rarely.
Hash tables are not so good if there are many insertions and deletions, or if table traversals are needed β in this case, AVL trees are better. In applications requiring frequent reading and writing of data, hash tables might not be a good solution. However, there are no hard and fast statistics β you must be a good software engineer to choose the relevant data structure.
Hashing is very slow for any operations which require the entries to be sorted, for example finding the minimum key. Data is inserted into the hash table array without any sort order and is scattered through the array with holes. The animation showed no real sequence of filling β some clusters formed due to collisions but there was no order. Hashing is not useful in these circumstances.
The important thing is how one data structure can be implemented in six different ways. As long as the interface remains the same, different internal implementations do not matter from the client perspective.
Sorting
Sorting means to put data in a certain order or sequence. Sorting has been discussed scattered through topics in this course but not as a separate topic. When traversing a binary search tree in in-order, the obtained data happens to be sorted. With min-heap, if elements are removed one by one, data is obtained in sorted order.
Sorting is so useful that in 80-90% of computer applications, sorting appears in one form or another. Normally, sorting and searching go together. Extensive research has been done on sorting, and very efficient algorithms have been developed. Vast mathematical analysis has been performed on these algorithms.
π Definition β Sorting: The process of arranging data in a specific order (typically ascending or descending) based on some comparison criteria.
π Example: Given array [20, 8, 5, 10, 7], sorting in ascending order produces [5, 7, 8, 10, 20]. The minimum number becomes the first element and the largest element becomes the last element.
Elementary Sorting Algorithms
The elementary sorting algorithms include Selection Sort, Insertion Sort, and Bubble Sort. These are called elementary because they are very simple and act as baseline algorithms for comparison with more efficient algorithms.
Selection Sort
Main idea:
- Find the smallest element
- Put it in the first position
- Find the next smallest element
- Put it in the second position
- Continue until you get to the end of the list
This technique searches the whole array and finds the smallest number. The smallest number is put on the first position while the previous element in this position is moved somewhere else. Then find the second smallest number and put it in the second position, again shifting the previous number. This activity is repeated until the array is sorted. This technique is called selection sort because we select elements for their sorted positions.
π‘ Why this matters: Selection sort provides a simple, intuitive approach to sorting that helps understand the fundamental concept of sorting algorithms before moving to more efficient methods.
β Key Takeaways
Hashing provides constant-time operations (insert, delete, find) but performance degrades with collisions. The three collision resolution strategies β linear probing, quadratic probing, and linked list chaining β each have different tradeoffs in time and memory usage. Hashing is most suitable for applications with many searches in stable tables (like compiler symbol tables and spell checkers) but is not suitable when many insertions/deletions are needed, table traversals are required, or sorted order is important. Sorting is fundamental to computer science, appearing in 80-90% of applications, and elementary algorithms like selection sort provide baseline understanding for studying more efficient sorting methods.
π§ Quick Revision Questions
-
What are the three collision resolution strategies demonstrated in the hashing animation, and how does each handle a collision when both parts of an array location are filled?
-
Why is hashing considered suitable for compiler symbol tables and spell checkers but unsuitable for operations requiring sorted output?
-
In selection sort, what is the main idea, and how does it arrange elements in ascending order step by step?
-
What is the hash function used in the linear probing animation, and how does it determine the array index for a given number?
-
When would AVL trees be a better choice than hash tables, and why?
π Lecture 44 β Data Structures Lecture No. 44
π Overview: This lecture continues the discussion of sorting algorithms, covering three elementary sorting methods: selection sort, insertion sort, and bubble sort. It explains their algorithms, code implementations, and time complexity analysis, concluding by introducing the concept of N logβ(N) algorithms as more efficient alternatives.
ποΈ Topics Covered
This lecture covers three elementary sorting algorithmsβselection sort, insertion sort, and bubble sortβwith detailed explanations, C++ code, and analysis of their O(NΒ²) time complexity. It also introduces the concept of N logβ(N) algorithms, including merge sort, quick sort, and heap sort, and compares their performance with the elementary sorts.
π Lecture Summary
Selection Sort
The main idea of selection sort is to repeatedly find the smallest element in the array and put it in its correct position. First, find the smallest element and swap it with the element in the first position. Then find the next smallest in the remaining elements and put it in the second position, continuing until the entire array is sorted.
π Definition β Selection Sort: An algorithm that repeatedly finds the smallest element from the unsorted portion and moves it to the beginning.
π Formula: Total searches = 1 + 2 + 3 + ... + N = N(N+1)/2 β O(NΒ²) β The total number of comparisons grows proportionally to NΒ².
π Example: Sorting array [19, 5, 7, 12] in ascending order:
- Step 1: Find smallest (5), swap with 19 β [5, 19, 7, 12]
- Step 2: Find smallest in remaining [19, 7, 12] (7), swap with 19 β [5, 7, 19, 12]
- Step 3: Find smallest in remaining [19, 12] (12), swap with 19 β [5, 7, 12, 19]
The findIndexMin function searches from a given start position to find the minimum element's index. The selectionSort function uses this to find the position, then swaps the minimum element with the current count position. This algorithm is an in-place sorting algorithm as it requires no additional storage.
π‘ Why this matters: Selection sort is simple but inefficient for large datasets because its time grows with NΒ².
Insertion Sort
The main idea of insertion sort is to build a sorted portion of the array on the left by inserting each new element into its proper position. Start by considering the first two elements and swapping if out of order. Then consider the third element and insert it into the proper position among the first three, and continue until all elements are processed.
π Definition β Insertion Sort: An algorithm that sorts by repeatedly taking the next element and inserting it into its correct position within the already sorted portion of the array.
π Formula: Total shifts = (2 + N)(N-1)/2 = O(NΒ²) β The worst-case number of shifts grows proportionally to NΒ².
π Example: Sorting array [19, 12, 5, 7]:
- Step 1: Take first two (19,12), 12 < 19, swap β [12, 19, 5, 7]
- Step 2: Take third element (5), shift 12 and 19 right, insert 5 at position 0 β [5, 12, 19, 7]
- Step 3: Take fourth element (7), shift 12 and 19 right, insert 7 between 5 and 12 β [5, 7, 12, 19]
The insertionSort function uses a variable val to hold the current element, then shifts larger sorted elements to the right to create space. When the inner loop exits, the value is inserted at arr[pos+1]. This shifting is the additional overhead that makes this algorithm O(NΒ²).
π‘ Why this matters: Insertion sort is efficient for small or nearly sorted datasets but poor for large unsorted arrays.
Bubble Sort
The main idea of bubble sort is to exchange neighboring items repeatedly until the largest item reaches the end of the array, then repeat for the remaining elements. Smaller elements "bubble up" to the top while larger elements sink to the bottom.
π Definition β Bubble Sort: An algorithm that repeatedly steps through the array, comparing adjacent elements and swapping them if they are in the wrong order.
π Formula: Total iterations = 1 + 2 + 3 + ... + N = N(N+1)/2 = O(NΒ²) β The number of comparisons grows proportionally to NΒ².
π Example: Sorting array [19, 12, 5, 7]:
- Pass 1: Compare (19,5) β swap β [5,19,12,7]; (19,12) β swap β [5,12,19,7]; (19,7) β swap β [5,12,7,19]
- Pass 2: Compare (5,12) β no swap; (12,7) β swap β [5,7,12,19]
- Pass 3: Compare (5,7) β no swap; array sorted
The bubbleSort function uses a swapped variable to track whether any swap occurred in a pass. The bound variable limits the inner loop to the unsorted portion. If no swap occurs in a pass, the while loop exits as the array is sorted.
π‘ Why this matters: Bubble sort can detect a sorted array early, but in the worst case, it is still O(NΒ²).
Summary
All three elementary algorithmsβselection sort, insertion sort, and bubble sortβare in place algorithms with time complexity proportional to NΒ². They are easy to understand and code but expensive for large datasets. The following table compares NΒ² and N logβ(N) for different values of N:
| N | NΒ² | N logβ(N) |
|---|---|---|
| 10 | 100 | 33.21 |
| 100 | 10,000 | 664.38 |
| 1,000 | 1,000,000 | 9,965.78 |
| 10,000 | 100,000,000 | 132,877.12 |
| 100,000 | 10,000,000,000 | 1,660,964.04 |
| 1,000,000 | 1,000,000,000,000 | 19,931,568.57 |
N logβ(N) Algorithms
N logβ(N) algorithms are significantly more efficient than NΒ² algorithms. These include merge sort, quick sort, and heap sort, which all fall under the divide and conquer category. The divide and conquer strategy splits the problem into smaller parts, solves each part separately, and then combines the results.
π Example of divide and conquer sorting:
- Unsorted array: [10, 12, 8, 4, 2, 11, 7, 5]
- Split into two parts: [10, 12, 8, 4] and [2, 11, 7, 5]
- Sort each part separately: [4, 8, 10, 12] and [2, 5, 7, 11]
- Merge the two sorted parts: [2, 4, 5, 7, 8, 10, 11, 12]
π‘ Why this matters: For large datasets, N logβ(N) algorithms can be thousands of times faster than NΒ² algorithms.
β Key Takeaways
The three elementary sorting algorithmsβselection sort, insertion sort, and bubble sortβare all in-place algorithms with O(NΒ²) time complexity, making them simple but inefficient for large datasets. The key differences are in their approach: selection sort finds the minimum and swaps, insertion sort builds a sorted portion by inserting each element, and bubble sort swaps adjacent pairs to bubble larger values to the end. For the exam, remember the NΒ² formulas for each algorithm and understand that N logβ(N) algorithms like merge sort, quick sort, and heap sort are far superior for large datasets, as shown by the comparison table where N=1,000,000 yields NΒ²=10ΒΉΒ² versus N logβ(N)β20 million.
π§ Quick Revision Questions
- What is the time complexity of selection sort, and what formula represents the total number of searches?
- In insertion sort, what is the main overhead that contributes to its O(NΒ²) time complexity?
- How does bubble sort determine when the array is completely sorted?
- Why are N logβ(N) algorithms considered better than NΒ² algorithms for large datasets?
- What is the divide and conquer strategy, and which sorting algorithms use it?
π Lecture 45 β Divide and Conquer, Mergesort, Quicksort, and Course Overview
π Overview: This lecture covers the divide and conquer strategy applied to sorting algorithms, focusing on mergesort and quicksort. It explains how these algorithms achieve O(n logβ n) time complexity compared to O(nΒ²) of elementary sorts, and concludes with a comprehensive overview of the entire data structures course.
ποΈ Topics Covered
The lecture introduces divide and conquer as a strategy to improve sorting efficiency, demonstrating how splitting a list into halves reduces time from nΒ² to approximately (n/2)Β²+(n/2)Β²+n. It covers the mergesort algorithm in detail including its recursive implementation, the mergeArrays procedure, and analysis showing O(n logβ n) time but O(n) extra space. Quicksort is introduced as a divide and conquer algorithm that uses partitioning around a pivot value. The lecture concludes with a course overview covering arrays, linked lists, stacks, queues, trees, AVL trees, threaded binary trees, union/find, table ADTs, skip lists, and hashing.
π Lecture Summary
Divide and Conquer
The divide and conquer strategy splits a list into two parts, sorts each part separately, and then merges them into a single sorted array. This approach significantly reduces sorting time compared to elementary algorithms.
For n=100, elementary sorting takes approximately nΒ² = 10000 time units. Using divide and conquer with insertion sort: (100/2)Β² + (100/2)Β² + 100 = 2500 + 2500 + 100 = 5100 β roughly half the time.
The subdivision can continue recursively: halves into quarters, quarters into eighths, until reaching single elements (n=1). This mirrors binary search where we subdivide an array until finding the target or reaching individual elements.
π‘ Why this matters: The divide and conquer strategy fundamentally improves sorting from O(nΒ²) to O(n logβ n), making it practical for large datasets.
Mergesort
Mergesort is a divide and conquer algorithm that:
- Splits the list in half
- Mergesorts the two halves recursively
- Merges the two sorted halves together
The algorithm involves three steps:
- If the number of items to sort is 0 or 1, return
- Recursively sort the first and second halves separately
- Merge the two sorted halves into a sorted group
π Definition β Mergesort: A recursive sorting algorithm that divides the list in half, sorts each half recursively, then merges the sorted halves.
mergeArrays
The mergeArrays function merges two sorted arrays into a single sorted temporary array. It uses three indexes: i for the first array, j for the second array, and k for the temporary array. At each step, the smaller of the two current elements is placed into the temporary array, and the corresponding index (i or j) is incremented along with k. When one array is exhausted, remaining elements from the other array are copied directly.
π Process: Compare a[i] and b[j], place the smaller into tmp[k], increment k and the appropriate index (i or j). Continue until one array is exhausted, then copy remaining elements.
π Example: Given arrays a = [3,5,15,28,30] and b = [6,10,14,22,43,50]:
- Compare 3 and 6 β place 3 in tmp
- Compare 5 and 6 β place 5 in tmp
- Compare 15 and 6 β place 6 in tmp
- Compare 15 and 10 β place 10 in tmp
- Continue until a is exhausted, then copy remaining b elements
- Result: tmp = [3,5,6,10,14,15,22,28,30,43,50]
Mergesort and Linked Lists
Mergesort also works with linked lists. The list is divided into two halves (knowing the size), each half is sorted recursively, and the sorted halves are merged together.
Mergesort Analysis
- Mergesort is O(n logβ n) time
- Merging two lists of size n/2 takes O(n) time
- There are logβ n levels of merging
- Mergesort is not an in-place sorting algorithm β it requires O(n) extra space for the temporary array
- This extra space requirement is a disadvantage compared to insertion sort and selection sort which are in-place
Quicksort
Quicksort is another divide and conquer algorithm based on partitioning the list around a pivot value. It is both O(n logβ n) and in-place (no extra array needed).
The process:
- Select a pivot element (e.g., the middle element)
- Swap it with the last element
- Use two indexes: low (starting from left, searching for element greater than pivot) and high (starting from right, searching for element less than pivot)
- Swap the elements at low and high when both stop
- Continue until low and high cross
- Swap the pivot element with the element at the crossing position
- Recursively quicksort the left part and right part
π Example: Array = [4,12,10,8,5,2,11,7,3], pivot = 5
- Swap pivot 5 with last element 3 β [4,12,10,8,3,2,11,7,5]
- low finds 12 (>5), high finds 2 (<5), swap β [4,2,10,8,3,12,11,7,5]
- low finds 10 (>5), high finds 3 (<5), swap β [4,2,3,8,10,12,11,7,5]
- low and high cross, swap pivot with element at crossing (8) β [4,2,3,5,10,12,11,7,8]
- Now 5 is in its final position; recursively sort left part [4,2,3] and right part [10,12,11,7,8]
Quicksort is considered one of the best general-purpose sorting algorithms.
Course Overview
The course began with arrays (fixed size limitation), then linked lists, stacks, and queues (implemented with arrays and linked lists). Stacks play a crucial role in computer runtime environments, and queues are essential for simulations.
Trees were introduced, particularly binary trees and AVL trees (balanced binary search trees). Threaded binary trees and union/find (up-tree) data structures were covered. The course emphasized Abstract Data Types (ADTs) β forming new data structures using existing ones. The table/dictionary ADT was implemented in six different ways, including skip lists. Hashing was discussed as a purely algorithmic procedure.
Important data structures not covered in detail include graphs, which are primarily important from an algorithmic perspective.
β Key Takeaways
The most critical concepts from this lecture are: (1) The divide and conquer strategy splits a problem into smaller subproblems, solves them recursively, and combines results β it reduces sorting time from O(nΒ²) to O(n logβ n). (2) Mergesort is O(n logβ n) but requires O(n) extra space for the temporary array, making it not in-place. (3) Quicksort is also O(n logβ n) but is an in-place algorithm that partitions around a pivot value. (4) Both mergesort and quicksort use recursion and are significantly faster than elementary O(nΒ²) sorts like insertion, selection, and bubble sort. (5) The course overview highlights that data structures and algorithms are complementary β algorithms bring along appropriate data structures, and choosing the right combination is a critical design skill for software engineers.
π§ Quick Revision Questions
- What are the three steps of the mergesort algorithm?
- Why is mergesort not considered an in-place sorting algorithm, and what is its space complexity?
- In quicksort, what happens when the low and high indexes cross each other during partitioning?
- How does the time complexity of divide and conquer (using insertion sort on halves) compare to plain insertion sort for n=100?
- What are the two key advantages of quicksort over mergesort?