π Lecture 20 β Overview of todayβs lecture
π Overview: This lecture covers the goals of a good memory allocator, the problem of memory fragmentation (internal and external), and implementation issues for memory allocators. It introduces implicit list-based allocators in detail, including how blocks are tracked, allocated, freed, and coalesced.
ποΈ Topics Covered
The lecture covers the goals of a good malloc/free system, including time performance and space utilization. It then explains internal and external fragmentation, and why fragmentation cannot be fully solved. Implementation issues such as knowing how much to free and tracking free blocks are discussed, followed by a detailed explanation of the implicit list-based allocator, including finding free blocks, allocating, freeing, and coalescing.
π Lecture Summary
Goals of Good malloc/free
The primary goals of a memory allocator are good time performance and good space utilization. For time, operations should ideally take constant time and certainly not linear time in the number of blocks. For space, user-allocated structures should make up a large fraction of the heap to minimize fragmentation. Other goals include good locality properties (allocations close in time should be close in space) and robustness (ability to check that a free pointer is valid and that memory references are to allocated space).
π Definition β Fragmentation: Poor memory utilization caused by the inability to use available memory efficiently.
Internal Fragmentation
Internal fragmentation is the difference between the block size and the payload size for a given block. It is caused by overhead from maintaining heap data structures, padding for alignment purposes, or explicit policy decisions (e.g., not splitting a block). Internal fragmentation depends only on the pattern of previous requests, making it easy to measure.
External Fragmentation
External fragmentation occurs when there is enough aggregate heap memory, but no single free block is large enough to satisfy a request. Unlike internal fragmentation, external fragmentation depends on the pattern of future requests and is therefore difficult to measure.
π‘ Why this matters: External fragmentation is a major challenge for allocators because it cannot be predicted or fully prevented.
Impossible to βsolveβ fragmentation
There is no "best" allocator because all designs involve tradeoffs. A theoretical result states that for any possible allocation algorithm, there exist streams of allocation and deallocation requests that defeat the allocator and force it into severe fragmentation. In practice, "pretty well" means about 20% fragmentation under many workloads.
Knowing How Much to Free
The standard method for knowing how much memory to free is to keep the length of a block in the word preceding the block. This word is often called the header field or header.
Keeping Track of Free Blocks
Four methods are described for keeping track of free blocks:
- Method 1: Implicit list β links all blocks using lengths.
- Method 2: Explicit list β uses pointers within the free blocks to link them.
- Method 3: Segregated free list β uses different free lists for different size classes.
- Method 4: Blocks sorted by size β uses a balanced tree (e.g., Red-Black tree) with pointers within each free block, using length as a key.
Method 1: Implicit List
The implicit list method requires identifying whether each block is free or allocated. This is typically done using an allocated flag stored in the block header.
Implicit List: Finding a Free Block
Three strategies are used to find a free block:
- First fit: Search the list from the beginning and choose the first free block that fits the request.
- Next fit: Like first-fit, but search the list from the location where the previous search ended.
- Best fit: Search the entire list and choose the free block with the closest size that fits the request.
Implicit List: Allocating in Free Block
When allocating in a free block, the allocator may need to split the block. Splitting divides the free block into two parts: one allocated block of the requested size and one remaining free block.
Implicit List: Freeing a Block
The simplest implementation of freeing a block only requires clearing the allocated flag:
void free_block(ptr p) { *p = *p & -2; }
Implicit List: Coalescing
Coalescing joins a freed block with the next and/or previous block if they are also free. This combines adjacent free blocks into a single larger free block, reducing external fragmentation.
Implicit List: Bidirectional Coalescing
To enable coalescing with the previous block, the allocator uses boundary tags (Knuth73). This replicates the size/allocated word at the bottom of free blocks, allowing traversal of the list backward. However, this requires extra space for the additional tag.
π Definition β Boundary tags: A technique that replicates the size and allocated status at both the top and bottom of a block, enabling bidirectional traversal for coalescing.
β Key Takeaways
The lecture establishes that memory allocators have primary goals of good time performance and space utilization, but fragmentation (both internal and external) is an inherent challenge that cannot be fully solved β any allocator can be defeated by a specific sequence of allocation and deallocation requests. Internal fragmentation depends on past requests and is easy to measure, while external fragmentation depends on future requests and is difficult to measure. Four methods for tracking free blocks are introduced, with the implicit list method being the most basic. Key operations in an implicit list allocator include finding a free block using first fit, next fit, or best fit; splitting during allocation; clearing the allocated flag during freeing; and coalescing with adjacent free blocks to reduce fragmentation. Boundary tags enable bidirectional coalescing but require extra overhead.
π§ Quick Revision Questions
- What are the two primary goals of a good memory allocator, and what are some additional goals?
- What is the difference between internal fragmentation and external fragmentation?
- Why is it impossible to have a "best" memory allocator?
- Describe the four methods for keeping track of free blocks in a memory allocator.
- What is coalescing in the context of an implicit list allocator, and how do boundary tags enable bidirectional coalescing?
π Lecture 21 β Overview of Todayβs Lecture
π Overview: This lecture completes the discussion of explicit free list allocators, covering LIFO vs. address-ordered freeing policies and segregated free lists. It then explores how real program allocation patterns (ramps, peaks, plateaus) can be exploited for performance, and introduces the fundamentals of garbage collection as a form of automatic memory management.
ποΈ Topics Covered
The lecture covers explicit free list base allocator details including freeing with LIFO policy and segregated free lists, then moves to exploiting allocation patterns of programs by exploiting peaks via arena allocators, and concludes with an introduction to garbage collection including reference counting, memory as a graph, and the assumptions garbage collectors make about pointers.
π Lecture Summary
Explicit Free Lists
Explicit free lists maintain a linked list data structure of only the free blocks in the heap, rather than marking free/allocated status in the block headers themselves. Each free block contains pointers to the previous and next free block in the list, allowing the allocator to traverse only free blocks rather than all blocks when searching for a suitable free block.
π Definition β Explicit Free List: A linked list that connects only the free blocks in the heap, where each free block contains a predecessor pointer and a successor pointer to other free blocks.
Allocating From Explicit Free Lists
When allocating from an explicit free list, the allocator searches the linked list of free blocks using a placement policy (e.g., first fit, best fit). Once a suitable block is found, it is removed from the free list. If the block is larger than needed, the remainder may be split and re-inserted into the free list as a smaller free block.
Freeing With Explicit Free Lists
The insertion policy determines where in the free list a newly freed block is placed. Two common policies exist:
- LIFO (last-in-first-out) policy: Insert the freed block at the beginning of the free list. This is simple and fast but can lead to fragmentation over time.
- Address-ordered policy: Insert freed blocks so that free list blocks are always in increasing address order (addr(pred) < addr(curr) < addr(succ)). This improves spatial locality and reduces fragmentation.
Freeing With a LIFO Policy
When freeing a block with LIFO policy, four cases arise depending on whether the adjacent blocks (previous and next in memory) are free or allocated:
- Case 1: a-a-a β Both neighbors are allocated. The freed block is simply inserted at the front of the free list as a single isolated free block.
- Case 2: a-a-f β The next block is free. The freed block is coalesced with the next free block, and the combined block is inserted at the front of the free list.
- Case 3: f-a-a β The previous block is free. The freed block is coalesced with the previous free block, which is already in the free list. The combined block remains at the previous block's position.
- Case 4: f-a-f β Both neighbors are free. The freed block is coalesced with both adjacent free blocks, and the combined block replaces them in the free list.
Explicit List Summary
- Comparison to implicit list: Allocation is linear time in the number of free blocks instead of total blocks β this is much faster when most memory is full.
- Disadvantages: Slightly more complicated allocate and free operations since blocks must be moved in and out of the list. Extra space is required for the links (2 extra words needed for each block).
- Main use: Linked lists are primarily used in conjunction with segregated free lists, where multiple linked lists are kept for different size classes or possibly for different types of objects.
Simple Segregated Storage
Simple segregated storage maintains a separate free list for each size class (e.g., 16-byte blocks, 32-byte blocks, 64-byte blocks). When a request comes in, the allocator goes directly to the appropriate size class list.
- No splitting: Blocks are never split; if a block of exact size is unavailable, the allocator may request more memory from the OS.
- Tradeoffs: This approach is very fast (O(1) allocation and free for fixed-size blocks) but can fragment badly because small blocks cannot be split to satisfy larger requests.
Segregated Fits
Segregated fits uses an array of free lists, each one for some size class. To allocate, the appropriate list is searched; to free a block, coalescing is performed and the block is placed on the appropriate list (optional).
- Tradeoffs: Faster search than sequential fits (log time for power-of-two size classes). Controls fragmentation better than simple segregated storage. However, coalescing can increase search times; deferred coalescing can help by postponing coalescing operations.
Known Patterns of Real Programs
Programs exhibit three common allocation patterns:
- Ramps: Accumulate data monotonically over time (e.g., logging systems that keep growing).
- Peaks: Allocate many objects, use them briefly, then free all of them (e.g., processing a single request or image).
- Plateaus: Allocate many objects, use them for a long time (e.g., data structures that persist throughout the program's lifetime).
Exploiting Peaks
Peak phases involve allocating a lot of memory, then freeing everything at once. An arena allocator can exploit this pattern by allocating memory in large chunks and simply resetting the pointer to the beginning when the phase ends.
- Advantages: Allocation is just a pointer increment (very fast), free is essentially free (no per-block work), and no wasted space for tags or list pointers.
Implicit Memory Management: Garbage Collection
Garbage collection is the automatic reclamation of heap-allocated storage β the application never has to explicitly call free. The memory manager automatically detects when memory is no longer reachable and reclaims it.
Garbage Collection
The memory manager must determine when memory can be freed. This requires certain assumptions about pointers:
- The memory manager can distinguish pointers from non-pointers (e.g., integers that happen to look like addresses).
- All pointers point to the start of a block (no pointers into the middle of allocated regions).
- Cannot hide pointers (e.g., by coercing them to an int and then back again), as the collector cannot track disguised references.
π‘ Why this matters: These assumptions are critical because the garbage collector must accurately identify which memory locations contain valid pointers to heap objects. If these assumptions are violated, the collector may free memory that is still in use or fail to reclaim memory that should be freed.
Reference Counting
Reference counting is a garbage collection algorithm where each object maintains a reference count β the number of pointers pointing to it.
- Algorithm:
- Each object has a "ref count" tracking how many pointers reference it.
- The ref count is incremented when a new pointer is set to point to the object.
- The ref count is decremented when a pointer is killed (e.g., overwritten or goes out of scope).
- When the ref count reaches zero, the object is freed.
π Definition β Reference Count: A counter associated with each object that tracks how many pointers currently reference that object; when it reaches zero, the object is reclaimed.
Problems with Reference Counting
Circular data structures always have ref count greater than zero because objects point to each other in a cycle, even when no external pointers reference the cycle. For example, if object A points to object B and object B points to object A, both have ref count β₯ 1, yet neither is reachable from the program's roots. This causes memory leaks since the garbage collector cannot reclaim cycles using reference counting alone.
π Example: Consider two objects, A and B, where A.next = B and B.prev = A. If the only pointer to A from the stack is removed, A and B still have ref counts of 1 each (pointing to each other), so neither is freed even though both are unreachable from the program.
Memory as a Graph
We view memory as a directed graph:
- Each block is a node in the graph.
- Each pointer is an edge in the graph.
- Root nodes are locations not in the heap that contain pointers into the heap (e.g., registers, locations on the stack, global variables).
Reachable memory includes all nodes that can be reached by following pointers starting from the root nodes. All memory that is not reachable from the roots is considered garbage and can be safely reclaimed.
Assumptions for Garbage Collection
The garbage collector relies on three key instructions/functions:
- is_ptr(p): Determines whether a value
pis a valid pointer (as opposed to data like an integer or float). - length(b): Returns the length of block
b, not including the header (to know where pointers might be located within the block). - get_roots(): Returns all root nodes (registers, stack locations, global variables) that contain potential pointers into the heap.
β Key Takeaways
The critical concepts from this lecture are: (1) Explicit free lists improve allocation speed by traversing only free blocks rather than all blocks, using either LIFO or address-ordered insertion policies; (2) Segregated free lists organize blocks by size class to further accelerate allocation and reduce fragmentation; (3) Real programs exhibit patterns (ramps, peaks, plateaus) that can be exploited β arena allocators leverage peak phases by using pointer increment allocation and batch-free operations; (4) Garbage collection automates memory reclamation by treating memory as a directed graph and reclaiming unreachable nodes, but relies on strict assumptions about pointer identification and structure; (5) Reference counting is simple but fails with circular data structures, motivating more sophisticated tracing collectors.
π§ Quick Revision Questions
- What are the four cases when freeing a block using LIFO policy, and how are they determined?
- What is the key advantage of explicit free lists over implicit free lists, and what is the cost?
- What is the difference between simple segregated storage and segregated fits, and what tradeoff does each make?
- How does an arena allocator exploit peak-phase allocation patterns, and what are its three advantages?
- Why does reference counting fail to reclaim circular data structures, and what assumption about pointers is essential for garbage collection to work correctly?
π Lecture 22 β Mark and Sweep Collecting
π Overview: This lecture covers a range of advanced operating system concepts, starting with memory management via the Mark and Sweep garbage collection algorithm. It then transitions into foundational OS topics including processes, address spaces, exceptions, process lifecycle (fork, exit, exec), synchronization primitives (semaphores, condition variables), deadlock conditions, thread usage paradigms, and basic scheduling with FCFS.
ποΈ Topics Covered
The lecture begins with Mark and Sweep garbage collection using depth-first traversal and length-based sweeping. It then defines an operating system from top-down and bottom-up views, covering time and space multiplexing. It lists types of operating systems and OS structures (monolithic, layered, micro-kernel). The ELF object file format is described with its header, program header table, and sections (.text, .data, .bss). Strong and weak symbols and linker rules are explained, along with static library creation. The lecture then moves to process definition, private address spaces, exceptions (asynchronous and synchronous), process creation, Unix SVR4 processes, Process Control Block (PCB), fork and exit system calls, zombies, exec, safe/unsafe trajectories, semaphores (for mutual exclusion and scheduling), condition variables, reentrant functions, deadlock definition and four necessary conditions, thread usage paradigms, and finally FCFS scheduling.
π Lecture Summary
Mark and Sweep Collecting
This section introduces garbage collection through the Mark and Sweep algorithm. It uses a depth-first traversal of the memory graph to mark all reachable objects, then sweeps through memory using lengths to find the next block of free or reclaimable memory. The algorithm identifies live objects by traversing from root references and collects dead objects by scanning the heap linearly.
π Definition β Mark and Sweep: A garbage collection algorithm that first marks all reachable objects via graph traversal (often depth-first) and then sweeps through memory to reclaim unmarked objects, using block lengths to navigate.
What is an operating system?
An operating system is described from two perspectives. The top-down view focuses on the interface and services provided to applications (e.g., resource abstraction). The bottom-up view focuses on managing hardware resources. Two key multiplexing strategies are used: time multiplexing (sharing CPU over time) and space multiplexing (sharing memory or storage concurrently).
π Definition β Time multiplexing: A resource sharing technique where the resource (like CPU) is allocated to different processes in time slices. Space multiplexing: A technique where the resource (like memory) is divided and allocated to multiple processes simultaneously.
Type of Operating Systems
Several categories of operating systems are listed: Main frame operating systems (e.g., IBM z/OS), Time-sharing systems (e.g., Unix), Multiprocessor operating systems (managing multiple CPUs), PC operating systems (e.g., Windows, Linux desktop), Real-time operating systems (guaranteeing timing constraints), and Embedded operating systems (for devices like phones, IoT).
OS Structure
Three OS architectures are presented:
- Monolithic Design: The entire OS runs in kernel space as a single large program, with all services tightly integrated.
- Layering: The OS is organized into hierarchical layers, where each layer provides services to the layer above and uses services from below.
- Micro-kernel: The kernel is minimal, providing only essential services (e.g., IPC, basic scheduling), while other services (e.g., file systems) run in user space.
ELF Object File Format
The Executable and Linkable Format (ELF) is described with its components: the Elf header (contains metadata like architecture, entry point), the Program header table (describes segments for loading), and three main sections: .text section (executable code), .data section (initialized global/static data), and .bss section (uninitialized global/static data, allocated at runtime).
π Definition β ELF: A standard file format for executables, object code, shared libraries, and core dumps, consisting of a header, program header table, and sections.
Strong and Weak Symbols
Program symbols are classified as either strong or weak. A strong symbol is typically a function or initialized global variable. A weak symbol is one declared without initialization or with special attributes (e.g., __attribute__((weak)) in GCC).
Linkerβs Symbol Rules
Three rules govern symbol resolution:
- Rule 1: A strong symbol can only appear once across all linked object files.
- Rule 2: A weak symbol can be overridden by a strong symbol of the same name.
- Rule 3: If there are multiple weak symbols, the linker can pick an arbitrary one.
π Formula/Rule: Rule 1: Strong β unique. Rule 2: Strong overrides weak. Rule 3: Multiple weak β any chosen.
Creating Static Libraries
This section describes the process of bundling multiple object files into a static library (e.g., .a file on Unix) using tools like ar, allowing programs to link against the library at compile time.
The Complete Picture
[This section heading appears but contains no substantive text beyond the heading itself in the provided lecture text. No summary is possible.]
Process
A process is defined as "a unit of activity characterized by the execution of a sequence of instructions, a current state, and an associated set of system instructions." It is the fundamental unit of work in an operating system.
Private Address Spaces
Each process is given its own private address space, meaning memory addresses used by one process do not refer to the same physical memory as those of another process. This provides isolation and protection between processes.
Asynchronous Exceptions (Interrupts)
These exceptions are caused by events external to the processor, such as I/O device completion, timer expiration, or hardware signals. They occur independently of the currently executing instruction.
Synchronous Exceptions
These exceptions are caused by the execution of an instruction itself. Three types are listed:
- Traps: Intentional exceptions (e.g., system calls).
- Faults: Potentially recoverable errors (e.g., page faults).
- Aborts: Unrecoverable errors (e.g., hardware failure).
Process Creation
The steps for creating a new process are:
- Assign a unique process identifier (PID).
- Allocate space for the process (memory for code, data, stack).
- Initialize process control block (PCB) with state and metadata.
Unix SVR4 Processes
[This section heading appears but contains no substantive text beyond the heading in the provided lecture text. No summary is possible.]
Process Control Block
The Process Control Block (PCB) is a data structure maintained by the OS for each process, containing key information:
- Process state: e.g., running, ready, waiting, halted.
- Priority: Used for scheduling decisions.
- Scheduling-related information: e.g., CPU burst times, wait times.
- Event: The event the process is waiting for (if blocked).
fork: Creating new processes
The fork() system call creates a new process. The child process is an almost exact copy of the parent. The function call is: int fork(void). It returns 0 to the child, the childβs PID to the parent, or -1 on failure.
π Example: A parent process calls fork(). After the call, two processes (parent and child) exist, both executing the next instruction after fork. The parent sees the childβs PID as the return value; the child sees 0.
exit: Destroying Process
The exit() system call terminates the calling process. The call is: void exit(int status). The status value is passed to the parent process (via wait()). The process becomes a zombie until reaped.
Zombies
- Idea: A zombie is a process that has exited (via
exit()) but whose exit status has not yet been read (reaped) by its parent. The processβs PCB is retained. - Reaping: The parent uses
wait()orwaitpid()to obtain the childβs exit status, after which the OS can fully deallocate the PCB. If the parent exits first, the orphaned process is adopted byinit(PID 1), which reaps it.
exec: Running new programs
The exec() family of system calls replaces the current processβs memory image with a new program. The call is: int execl(char *path, char *arg0, char *arg1, ..., 0). It loads the program at path and passes the arguments. If successful, it never returns; on failure, it returns -1.
Safe and Unsafe Trajectories
[This section heading appears but contains no substantive text beyond the heading in the provided lecture text. No summary is possible. In context, safe/unsafe trajectories relate to deadlock avoidance, where a safe trajectory avoids deadlock states.]
Semaphores
A semaphore is a synchronization primitive that is higher level than locks. It was invented by Dijkstra in 1968 as part of the THE operating system. A semaphore is an integer variable accessed through two atomic operations: P() (wait) and V() (signal).
Two uses of semaphores are identified:
- Mutual exclusion: Ensuring only one thread accesses a critical section at a time (binary semaphore).
- Scheduling constraints: Coordinating the order of execution (e.g., thread A must complete before thread B starts, using a counting semaphore).
π Definition β Semaphore: A synchronization primitive with a counter, supporting atomic wait() (decrement, block if zero) and signal() (increment, wake a blocked thread if any).
Condition variables
A condition variable is a queue of threads waiting for something to happen inside a critical section. It provides three operations:
- Wait(): Release the lock, go to sleep, and re-acquire the lock when woken.
- Signal(): Wake up one waiter, if any.
- Broadcast(): Wake up all waiters.
Condition variables are always used with a mutex lock to protect the shared state being checked.
π Definition β Condition variable: A synchronization primitive that allows threads to wait for a specific condition to become true within a critical section, releasing the lock while sleeping.
Reentrant Functions
[This section heading appears but contains no substantive text beyond the heading in the provided lecture text. No summary is possible. A reentrant function is one that can be safely called again before a previous invocation has completed, typically by not relying on static global data.]
Formal definition of a deadlock
A deadlock is formally defined: "A set of processes is deadlocked if each process in the set is waiting for an event that only another process in the set can cause." This means processes hold some resources while waiting for others held by each other, creating a cycle.
Conditions for deadlock
All four conditions must hold simultaneously for a deadlock to occur:
- Mutual exclusion: Resources cannot be shared; only one process can use a resource at a time.
- No preemption: Resources cannot be forcibly taken away from a process; only the holder can release voluntarily.
- Hold and wait: A process holds at least one resource while waiting to acquire additional resources.
- Circular wait: A cycle exists in the resource allocation graph, where each process in the cycle waits for a resource held by the next.
π Formula (Rule): Without all four conditions, deadlock cannot occur.
Paradigms of thread usage (from Hauser et al paper)
This section lists several common patterns for using threads:
- Defer work: Offload work from a critical path.
- General pumps: Threads that continuously process items from a queue.
- Slack processes: Threads that run when idle, doing background work.
- Sleepers: Threads that sleep until a specific time or event.
- One-shots: Threads that perform a single task and then terminate.
- Deadlock avoidance: Threads designed to prevent deadlock via ordering.
- Rejuvenation: Periodic restart to avoid accumulating errors.
- Serializers: Threads that handle sequential access to shared resources.
- Encapsulated fork: Forking a thread to isolate risky code.
- Exploiting parallelism: Using multiple threads to speed up computation on multiple cores.
Scheduling: First come first served (FCFS or FIFO)
This is the simplest scheduling algorithm: Run jobs in the order that they arrive. It uses a FIFO queue. It is non-preemptive; once a job starts, it runs to completion or until it blocks.
π Example: Three processes arrive at time 0, 2, 4 with burst times of 5, 3, 1. FCFS schedules them in arrival order: P1 (0-5), P2 (5-8), P3 (8-9). Average waiting time = (0+3+4)/3 = 2.33 time units.
β Key Takeaways
The Mark and Sweep algorithm uses depth-first traversal to mark live objects and linear sweeping to reclaim memory. Processes are isolated via private address spaces, and exceptions are classified as asynchronous (interrupts) or synchronous (traps, faults, aborts). Fork creates a copy of the process, exit terminates it (creating a zombie until reaped), and exec loads a new program. Semaphores enable both mutual exclusion and scheduling constraints, while deadlock requires all four conditions: mutual exclusion, no preemption, hold and wait, and circular wait. FCFS scheduling is simple but can cause long average waiting times (convoy effect).
π§ Quick Revision Questions
- Explain the two main phases of the Mark and Sweep garbage collection algorithm. How does depth-first traversal work during the mark phase?
- What are the four conditions that must all hold for a deadlock to occur? Give a real-world example showing each condition.
- Describe the difference between
fork()andexec()system calls. What happens to the processβs address space in each case? - How does a semaphore differ from a condition variable? Provide one use case for each.
- Why can a strong symbol appear only once across linked object files, but multiple weak symbols are allowed? What rule does this relate to in the linker?