π Lecture 23 β Overview of todayβs lecture
π Overview: This lecture introduces the fundamental goals and mechanisms of OS memory management. It explores how memory is allocated among competing processes, the historical evolution from simple techniques like fixed and variable partitioning to modern virtual memory, and the critical issues of protection, relocation, and fragmentation.
ποΈ Topics Covered
The lecture covers the goals of OS memory management, the core questions regarding memory management, the concept of multiprogramming with linker-loaders and swapping, the use of virtual addresses for multiprogramming, the memory hierarchy, and the two old techniques of fixed partitioning and variable partitioning, including their mechanics and the problem of fragmentation.
π Lecture Summary
Goals of OS memory management
The primary goals are to allocate scarce memory resources among competing processes to maximize memory utilization and system throughput, and to provide isolation between processes so that one process cannot interfere with another's memory. The tools used to achieve these goals include base and limit registers, swapping, paging (with page tables and TLBs), segmentation (with segment tables), page fault handling (which enables virtual memory), and the policies that govern the use of these mechanisms.
π‘ Why this matters: Understanding these goals is foundational; every memory management technique is designed to solve one or more of these core problems.
Our main questions regarding Memory Management
The three central questions that memory management must answer are: How is protection enforced? How are processes relocated? And how is memory partitioned?
Todayβs desktop and server systems
The basic abstraction for memory management in modern systems is virtual memory (VM). VM enables programs to execute without requiring their entire address space to be resident in physical memory, allowing a program to run on machines with less RAM than it "needs." This works because many programs don't need all of their code or data at once (e.g., branches they never take). Furthermore, virtual memory isolates processes from each other, as one process cannot name the addresses visible to others; each process has its own isolated address space. Virtual memory requires hardware and OS support, including MMUs, TLBs, page tables, and page fault handling, and is typically accompanied by swapping and at least limited segmentation.
Multiprogramming: Linker-loader
Can multiple programs share physical memory without hardware translation? Yes. When a program is copied into memory, its addresses (for loads, stores, jumps) are changed to use the addresses of where the program lands in memory. This relocation is performed by a linker-loader (e.g., UNIX ld). The compiler generates each .o file with code that starts at location 0. To create an executable, the linker-loader scans each .o, changing addresses to point to where each module goes in the larger program.
- Swapping is a technique where a program's entire state (including its memory image) is saved to disk, allowing another program to run. The first program can be swapped back in later and restarted right where it was.
- With multiple processes/jobs in memory at once (to overlap I/O and computation), memory management requirements include: protection (restrict which addresses processes can use), fast translation (memory lookups must be fast despite the protection scheme), and fast context switching (updating memory hardware must be quick when switching between jobs).
π Definition β Linker-loader: A program that combines multiple object files into a single executable, adjusting addresses (relocation) to reflect where the program will reside in physical memory. π Definition β Swapping: Saving a process's entire memory image to disk and restoring it later, allowing the CPU to run other processes in the meantime.
Virtual addresses for multiprogramming
To make it easier to manage memory of multiple processes, processes use virtual addresses. These are independent of the location in physical memory (RAM) where the referenced data lives. The OS determines the location in physical memory. Instructions issued by the CPU reference virtual addresses (e.g., pointers, arguments to load/store instructions, the PC). These virtual addresses are translated by hardware into physical addresses (with some setup from the OS). The set of virtual addresses a process can reference is its address space. This is not yet paging or virtual memoryβonly that the program issues addresses in a virtual address space that must be "adjusted" to reference memory.
π Definition β Virtual Address: An address in a process's logical address space that is independent of the physical memory location where the data is stored.
Memory Hierarchy
The memory hierarchy is based on two principles: the smaller the amount of memory needed, the faster that memory can be accessed; and the larger the amount of memory, the cheaper per byte. This works because programs exploit locality. Temporal locality means a program will reference the same locations as accessed in the recent past. Spatial locality means a program will reference locations near those accessed in the recent past.
π Definition β Temporal Locality: The tendency of a processor to access the same memory locations repeatedly over a short period of time. π Definition β Spatial Locality: The tendency of a processor to access memory locations that are near previously accessed locations.
Old technique #1: Fixed partitions
In this technique, physical memory is broken up into fixed partitions. Partitions may have different sizes, but the partitioning never changes. The hardware requirement is a base register and a limit register. The physical address is calculated as physical address = virtual address + base register. Base register is loaded by the OS when it switches to a process. Protection is provided by checking if the physical address exceeds the limit register's value. The main advantage is simplicity. However, it suffers from internal fragmentation (the available partition is larger than what was requested) and external fragmentation (two small partitions are left, but one big job cannot fit).
π Definition β Base Register: A hardware register that holds the starting physical address of a process's memory partition. π Definition β Limit Register: A hardware register that holds the size of a process's memory partition, used for protection. π Definition β Internal Fragmentation: Wasted memory space within a allocated partition because the partition is larger than the process needs.
Old technique #2: Variable partitions
The next step is to break up physical memory into partitions dynamically, tailoring partitions to programs. The hardware requirements are still a base register and a limit register. The physical address is calculated as physical address = virtual address + base register. Protection is again provided by comparing against the limit register. The advantage is no internal fragmentation, as the partition size is exactly what the process needs. The main problem is external fragmentation: as jobs are loaded and unloaded, holes are left scattered throughout physical memory.
π Definition β External Fragmentation: Wasted memory space between allocated partitions, where the total free memory is sufficient but not contiguous.
Dealing with fragmentation
One method to deal with fragmentation is to swap a program out, then reload it adjacent to another process, and adjust its base register.
β Key Takeaways
The three core questions of memory management are protection, relocation, and partitioning. While modern systems use virtual memory, historical techniques like fixed and variable partitioning illustrate fundamental trade-offs: fixed partitions are simple but cause internal fragmentation, while variable partitions eliminate internal fragmentation but cause external fragmentation. The base and limit registers are a simple hardware mechanism for relocation and protection. Multiprogramming requires memory management to prevent processes from interfering with each other, achieved initially through linker-loaders and swapping. The memory hierarchy exploits locality to balance speed and cost.
π§ Quick Revision Questions
- What are the three main questions that memory management must answer?
- What is the difference between internal fragmentation and external fragmentation?
- How do base and limit registers provide both relocation and protection in a fixed partition system?
- What is the role of a linker-loader in a multiprogramming system without hardware address translation?
- Explain the difference between temporal and spatial locality in the context of the memory hierarchy.
π Lecture 24 β Paging
π Overview: This lecture introduces paging as a modern memory management technique that solves external fragmentation by using fixed-sized units. It covers address translation mechanics, page table structures, page fault handling, and multi-level page tables, explaining how virtual addresses are mapped to physical memory frames.
ποΈ Topics Covered
The lecture covers paging as a technique to solve external fragmentation, the user's perspective of virtual address space and virtual-to-physical mapping, address translation mechanics including VPN and PFN, page table structures and page table entries with their control bits, paging advantages and disadvantages including internal fragmentation, page faults and their handling via OS exception handlers, and multi-level page tables as a solution to large page table memory requirements.
π Lecture Summary
Modern technique: Paging
Paging solves the external fragmentation problem by using fixed sized units in both physical and virtual memory. External fragmentation occurs when free gaps exist between allocated memory chunks, unlike internal fragmentation where free gaps exist because the process doesn't need all of an allocated chunk.
π Definition β Paging: A memory management scheme that divides both virtual and physical memory into fixed-size blocks called pages (virtual) and frames (physical).
Userβs perspective
Processes view memory as a contiguous address space from bytes 0 through N, called the virtual address space (VAS). In reality, virtual pages are scattered across physical memory frames β not contiguous. This virtual-to-physical mapping is invisible to the program. Protection is provided because a program cannot reference memory outside of its VAS; the virtual address 0x356AF maps to different physical addresses for different processes. For now, assume all pages of the address space are resident in memory β no "page faults."
Address translation
Translating virtual addresses requires two parts: virtual page number (VPN) and offset. The VPN is an index into a page table, and the page table entry contains a page frame number (PFN). The physical address is PFN concatenated with offset.
π Definition β Page table: A data structure managed by the OS that maps virtual page numbers (VPN) to page frame numbers (PFN). VPN is simply an index into the page table.
π Formula: Physical Address = PFN::offset (concatenation of page frame number and offset)
π Example: Assume 32-bit addresses with 4KB page size (4096 bytes, 2^12 bytes). VPN is 20 bits long (2^20 VPNs), offset is 12 bits long. Translate virtual address 0x13325328: VPN is 0x13325, offset is 0x328. Assume page table entry 0x13325 contains value 0x03004 (PFN = 0x03004). Physical address = PFN::offset = 0x03004328.
Page Table Entries (PTEs)
PTEs control mapping with specific bits: the valid bit says whether the PTE can be used and checks if a virtual address is valid; the referenced bit says whether the page has been accessed (set when read or written); the modified bit says whether the page is dirty (set when a write occurs); the protection bits control allowed operations (read, write, execute); and the page frame number determines the physical page start address.
π Definition β Valid bit: A bit in PTE indicating whether the virtual address is valid and can be used.
π Definition β Modified bit: A bit in PTE indicating whether the page has been written to (dirty).
π‘ Why this matters: These control bits allow the OS to implement virtual memory, enforce protection, and track page usage for swapping decisions.
Paging advantages
Physical memory is easy to allocate from a free list of frames β to allocate a frame, just remove it from the free list. External fragmentation is not a problem because managing variable-sized allocations is complex. Paging leads naturally to virtual memory where the entire program need not be memory resident, using page faults with the "valid" bit. However, paging was originally introduced to deal with external fragmentation, not to allow programs to be partially resident.
Paging disadvantages
Internal fragmentation can still occur β a process may not use memory in exact multiples of pages. There is a memory reference overhead of 2 references per address lookup (page table, then memory). The solution is using a hardware cache called translation lookaside buffer (TLB) to absorb page table lookups. Memory required to hold page tables can be large β need one PTE per page in virtual address space. A 32-bit address space with 4KB pages requires 2^20 PTEs = 1,048,576 PTEs, with 4 bytes/PTE = 4MB per page table. OS typically has separate page tables per process β 25 processes = 100MB of page tables. Solution: page the page tables.
π Formula: Page table size = (Address space size / Page size) Γ PTE size
π Example: For 32-bit AS with 4KB pages and 4-byte PTE: (2^32 / 2^12) Γ 4 = 2^20 Γ 4 = 4MB per page table.
Page Faults (like βCache Missesβ)
What if an object is on disk rather than in memory? The page table entry indicates the virtual address is not in memory. An OS exception handler is invoked to move data from disk into memory β the current process suspends while others can resume, and the OS has full control over placement.
Servicing a page fault involves:
- Processor signals controller to read block of length P starting at disk address X, storing at memory address Y
- Read occurs via Direct Memory Access (DMA) under I/O controller control
- I/O controller signals completion via interrupt, OS resumes suspended process
π Definition β Page fault: An interrupt that occurs when a program attempts to access a memory page that is not currently in physical memory, requiring the OS to load it from disk.
π Definition β DMA (Direct Memory Access): A method that allows I/O controllers to transfer data directly between disk and memory without CPU intervention.
π Example: Suppose page size is 4 bytes. A small page size means lots of space taken up with page table entries (like VAX had 512-byte pages). A large page size wastes unused space inside the page (internal fragmentation).
Multi-Level Page Tables
Given 4KB page size, 32-bit address space, and 4-byte PTE, we would need a 4MB page table (2^20 Γ 4 bytes). The common solution is multi-level page tables, e.g., a 2-level table. Level 1 table has 1024 entries, each pointing to a Level 2 page table. Level 2 table has 1024 entries, each pointing to a physical page.
π Definition β Multi-level page table: A hierarchical page table structure where the virtual address is divided into multiple indices, with outer levels pointing to inner level tables, reducing memory usage for sparse address spaces.
π Example: With 4K pages (12-bit offset), 4 bytes/PTE, and master page table fitting in one page (4K/4 bytes = 1K entries), we have 1K secondary page tables. The virtual address has three parts: Master page number, Secondary page number, and Offset.
Addressing Page Tables
Page tables can be stored in physical memory (easy to address, no translation required, but allocated page tables consume memory for lifetime of VAS) or in virtual memory (OS virtual address space) β cold page table pages can be paged out to disk, but addressing page tables requires translation. To stop recursion, do not page the outer page table (called wiring). If we're going to page the page tables, we might as well page the entire OS address space, but we need to wire special code and data (fault, interrupt handler).
π Definition β Wiring: The process of marking certain pages (like outer page tables and interrupt handlers) as non-pageable, ensuring they remain in physical memory.
π‘ Why this matters: Wiring prevents infinite recursion when the OS tries to handle a page fault but the page table itself is paged out.
β Key Takeaways
Paging solves external fragmentation by using fixed-size pages and frames, with the OS maintaining page tables to map virtual page numbers to physical frame numbers. Address translation requires splitting virtual addresses into VPN and offset, with page table entries containing control bits (valid, referenced, modified, protection) for memory management and protection. Page faults occur when accessed pages are not in memory, requiring the OS to load them from disk while suspending the process. Multi-level page tables reduce memory overhead by using hierarchical structures, where outer level tables point to inner tables, and wiring prevents recursive page faults when paging the page tables.
π§ Quick Revision Questions
- What are the two main parts of a virtual address in paging, and how is the physical address constructed from them?
- List and explain the five control bits found in a typical page table entry (PTE).
- What is internal fragmentation, and why does paging still suffer from it despite solving external fragmentation?
- Explain the three steps involved in servicing a page fault, including the role of DMA.
- Why are multi-level page tables needed, and how do they reduce memory overhead compared to a single-level page table?
π Lecture 25 β Overview of todayβs lecture
π Overview: This lecture introduces segmentation as an alternative memory management technique that partitions address spaces into logical units. It then explores the combination of segmentation and paging, the challenges of efficient address translation, and the critical role of the Translation Lookaside Buffer (TLB) in speeding up virtual-to-physical address lookups.
ποΈ Topics Covered
The lecture covers segmentation as a logical partitioning method, contrasts it with paging, and discusses its hardware support, pros, and cons. It then explains the combination of segmentation and paging, including the translation process and its advantages and disadvantages. The lecture concludes with a discussion on efficient translations, caching principles, and the detailed operation of the Translation Lookaside Buffer (TLB), including different mapping schemes.
π Lecture Summary
Segmentation
- Paging mitigates various memory allocation complexities (e.g., fragmentation) by viewing an address space as a linear array of bytes, dividing it into pages of equal size (e.g., 4KB), and using a page table to map virtual pages to physical page frames.
- Segmentation partitions an address space into logical units such as stack, code, heap, subroutines. A virtual address is structured as
<segment #, offset>.
Whatβs the point?
- More βlogicalβ: Without segmentation, a linker takes independent modules and organizes them. Segmentation treats them as truly independent.
- Facilitates sharing and reuse: A segment (e.g., a subroutine) is a natural unit for sharing.
- A natural extension of variable-sized partitions: A variable-sized partition is 1 segment per process; segmentation allows many segments per process.
Hardware support
- A segment table holds multiple base/limit pairs, one per segment. Segments are named by a segment number, which is used as an index into the table. The offset of a virtual address is added to the base address of the segment to yield the physical address.
Segmentation pros & cons
- Pros: + Efficient for sparse address spaces. + Easy to share whole segments (e.g., code segment). + Protection modes can be added (e.g., code segment as read-only).
- Cons: 1. Complex memory allocation (still needs first fit, best fit, etc., and reshuffling to coalesce free fragments).
- Linux Example: 1 kernel code segment, 1 kernel data segment, 1 user code segment, 1 user data segment, N task state segments, 1 local descriptor table segment. All of these segments are paged.
Segmentation and Paging
- Can combine segmentation and paging. The x86 supports both.
- Use segments to manage logically related units (module, procedure, stack). Segments vary in size but are usually large (multiple pages).
- Use pages to partition segments into fixed-size chunks. This makes segments easier to manage (βpageableβ)βmove page portions instead of whole segments. Only need to allocate page table entries for pieces of segments that have been allocated. Tends to be complex.
Segmentation with paging translation
- The translation process involves using the segment table to get a base address for a page directory, then using the page directory and page table to find the physical page frame.
Segmentation with paging translation Pros & Cons
- Pros: + Only need to allocate as many page table entries as needed (sparse address spaces are easy). + Easy memory allocation. + Share at segment or page level.
- Cons: - Pointer per page (typically 4KB-16KB pages today). - Page tables need to be contiguous. - Two lookups per memory reference (plus the segment lookup, making it more).
Efficient Translations
- The original page table scheme already doubles the cost of memory lookups (one lookup into the page table, another to fetch the data).
- Two-level page tables triple the cost (two lookups into page tables, a third to fetch the data), assuming the page table is in memory.
- How to use paging with lookups costing about the same as fetching from memory? 1. Cache translation in hardware. 2. Translation Lookaside Buffer (TLB). 3. TLB managed by Memory Management Unit (MMU).
Integrating VM and Cache
- Most Caches are βPhysically Addressedβ: Accessed by physical addresses. This allows multiple processes to have blocks in cache at the same time and share pages. The cache does not need to be concerned with protection issues (access rights are checked as part of address translation).
- Perform Address Translation Before Cache Lookup: This could involve a memory access itself (of the PTE). Page table entries can also become cached.
Caching review
- A cache is a copy that can be accessed more quickly than the original. The idea is to make the frequent case efficient. Caching underlies many techniques (translations, memory locations, pages, file blocks, etc.).
- Generic Issues in Caching:
- Cache hit: Item is in the cache.
- Cache miss: Item is not in the cache; full operation is needed.
- Effective access time = P(hit) * cost of hit + P(miss) * cost of miss.
- Key questions: How to find if an item is in the cache (hit)? How to choose what to replace on a miss (replacement policy)? How to keep the cache copy consistent with the real version (consistency)?
Speeding up Translation with a TLB
- βTranslation Lookaside Bufferβ (TLB): A small hardware cache in the MMU. It maps virtual page numbers to physical page numbers and contains complete page table entries for a small number of pages.
- The TLB is a hardware table of frequently used translations, avoiding the page table lookup in the common case. Typically on-chip, with an access time of 2-5ns (vs. 30-100ns for main memory).
How do we tell if needed translation is in TLB?
- Sequential order: Search the table sequentially.
- Direct mapped: Restrict each virtual page to use a specific slot in the TLB. For example, use upper bits of the virtual page number to index the TLB, then compare against lower bits to check for a match. A problem arises if two pages (e.g., program counter and stack) conflict for the same slot. This requires picking a hash function to minimize conflicts (e.g., using a selection of high-order and low-order bits as the index).
- Set associativity: Arrange the TLB (or cache) as N separate banks and do a simultaneous lookup in each bank. This is called an "N-way set associative cache". More set associativity reduces the chance of thrashing, as translations can be stored in either bank.
- Fully associative: A translation can be stored anywhere in the TLB, so all entries are checked in parallel.
π Definition β Translation Lookaside Buffer (TLB): A small, fast hardware cache in the MMU that stores recent virtual-to-physical address translations to speed up memory access.
π Definition β Direct Mapped TLB: A TLB structure where each virtual page number can only map to one specific slot in the TLB, determined by a hash of its address bits.
π Definition β N-way Set Associative TLB: A TLB organized into N banks, where a virtual page can be stored in any one of the N banks indexed by a subset of its address bits, allowing simultaneous lookups.
π Example: Direct mapped TLB conflict: If the program counter (PC) and stack pointer (SP) access different virtual pages that map to the same TLB slot (due to a simple indexing scheme), accessing them alternately will cause a TLB miss every time, resulting in thrashing. A better hash function (e.g., mixing high and low bits) can reduce this. A 2-way set associative TLB would allow both translations to be stored simultaneously in the two banks.
π‘ Why this matters: The TLB is a crucial performance feature in modern operating systems and CPUs. Without it, every memory access would require multiple slow memory references for page table lookups, dramatically reducing performance. The design of the TLB (direct mapped, set associative, fully associative) is a trade-off between hardware complexity, speed, and the likelihood of translation misses (thrashing).
β Key Takeaways
Segmentation provides a more logical view of memory by partitioning it into units like code and stack, facilitating sharing and protection, though it suffers from complex allocation. Combining segmentation with paging leverages the benefits of both, allowing easy management of large logical units while handling them with small, fixed-size pages in physical memory, though this increases translation overhead. To mitigate the performance cost of multi-level address translation, the TLB caches recent page table entries in the MMU. The design of the TLBβwhether direct-mapped, set-associative, or fully associativeβis a critical trade-off between hardware cost and the likelihood of translation misses, which directly impacts system performance.
π§ Quick Revision Questions
- How does a virtual address differ structurally between a pure paging system and a segmentation system?
- What are two key advantages of combining segmentation with paging?
- What is the primary purpose of the Translation Lookaside Buffer (TLB)?
- Explain the difference between a "direct-mapped" TLB and a "fully associative" TLB in terms of where a translation can be stored.
- What is the main problem with a direct-mapped TLB when two frequently used pages map to the same slot, and how does set associativity solve this?
π Lecture 26 β Set Associative and Fully Associative Caches, Demand Paging, Page Replacement Algorithms
π Overview: This lecture explores how hardware caches (set associative and fully associative) and software page replacement work together in memory management. It explains the mechanics of demand paging, page fault handling, and various page replacement algorithms, including their trade-offs between performance and complexity. Understanding these concepts is critical for grasping how virtual memory systems balance speed, memory utilization, and fairness.
ποΈ Topics Covered
The lecture begins by contrasting set associative and fully associative caches, discussing TLB design and replacement policies. It then covers TLB consistency with page tables during context switches and page table entry (PTE) fields. The discussion shifts to paged virtual memory, demand paging, page faults, and program loading. Finally, it details page replacement algorithms: Belady's optimal, FIFO, LRU, approximate LRU, and the Clock (NRU) algorithm, including global vs. local replacement strategies.
π Lecture Summary
Set associative cache
A set associative cache is a compromise between direct-mapped and fully associative caches. It divides cache into sets, where each set contains multiple "ways" (cache lines). An address maps to a specific set, but can occupy any way within that set, requiring fewer comparators than fully associative designs.
π‘ Why this matters: Set associativity reduces conflict misses compared to direct mapping while keeping hardware complexity manageable.
Fully associative TLB
A fully associative cache has one element per bank with one comparator per bank, allowing any memory address to be stored in any cache line. TLBs (Translation Lookaside Buffers) are typically small and fully associative, while larger hardware caches use direct-mapped or low-degree set associative designs.
- For replacement in set associative or fully associative caches, hardware often chooses randomly (simple and fast), while software page replacement uses more sophisticated algorithms.
- Tradeoff: Spend CPU cycles on smarter replacement to improve cache hit rate.
Consistency between TLB and page tables
- On context switch, the entire TLB must be invalidated because the new program will bring in new translations.
- When translation tables change (e.g., page moved between memory and disk), the corresponding TLB entry must be invalidated.
- Basic mechanism using TLB "present" (valid) bit:
- If present: pointer to page frame in memory
- If not present: use page table in memory
- Hardware traps to OS on reference not in TLB
- OS loads page table entry into TLB and continues thread
- All of this is transparent to the job.
Reminder: Page Table Entries (PTEs)
- Valid bit: Says whether the PTE can be used; checked each time a virtual address is used
- Referenced bit: Says whether the page has been accessed; set when a page is read or written
- Modified bit (dirty bit): Says whether the page is dirty; set when a write occurs
- Protection bits: Control which operations are allowed (read, write, execute)
- Page Frame Number (PFN): Determines the physical page; physical page start address = PFN
Paged virtual memory
The full (used) address space exists on secondary storage (disk) in page-sized blocks. The OS uses main memory as a (page) cache. When a page is needed, it is transferred to a free page frame. If no free frames exist, a page must be evicted (evicted pages go to disk only if dirty). All of this is transparent to the application, managed by hardware and OS. This is traditionally called paged virtual memory.
Page faults
When a process references a virtual address in a page that has been evicted:
- When the page was evicted, the OS set the PTE as invalid and stored the disk location in a data structure (like a page table but holding disk addresses)
- When the process accesses the page, the invalid PTE causes an exception (page fault/interrupt)
- The OS runs the page fault handler:
- Uses the "like a page table" data structure to locate the page on disk
- Reads the page into a physical frame, updates PTE to point to it and sets it valid
- OS restarts the faulting process
Demand paging
Demand paging means pages are only brought into main memory when they are referenced. Only the code/data that is needed (demanded) by a process needs to be loaded. Few systems try to anticipate future needs (OS "crystal ball module" is notoriously ineffective). However, clustering is common: the OS keeps track of pages that should come and go together, bringing them all in when one is referenced.
How do you "load" a program?
- Create process descriptor (process control block)
- Create page table
- Put address space image on disk in page-sized chunks
- Build page table (pointed to by process descriptor):
- All PTE valid bits set to false
- An analogous data structure indicates the disk location of the corresponding page
- When process starts executing:
- Instructions immediately fault on both code and data pages
- Faults taper off as necessary code/data pages enter memory
Page replacement
When reading in a page:
- If there are free page frames, grab one
- If not, must evict something else (this is page replacement)
Page replacement algorithms:
- Try to pick a page that won't be needed in the near future
- Try to pick a page that hasn't been modified (saving disk write)
- OS typically keeps a pool of free pages to avoid inevitable evictions
- OS also keeps some "clean" pages around so evictions don't require writes (accomplished by pre-writing when idle)
How does it all work? Locality!
- Temporal locality: Locations referenced recently tend to be referenced again soon
- Spatial locality: Locations near recently referenced locations are likely to be referenced soon
Locality means paging can be infrequent β once paged in, something will be used many times. However, this depends on: degree of locality in the application, page replacement policy and reference pattern, and amount of physical memory vs. application "footprint" or "working set".
Evicting the best page
Goal: Reduce fault rate by selecting the best victim page. The best page to evict is one that will never be touched again (impossible to predict). Belady's proof: evicting the page that won't be used for the longest period of time minimizes page fault rate.
#1: Belady's Algorithm
π Definition β Belady's Algorithm (OPT): The optimal page replacement algorithm that evicts the page that will not be used for the longest time in the future.
π Formula: Evict page with maximum "next use time" β minimizes page fault rate
- Provably optimal: lowest fault rate
- Problem: Impossible to predict the future
- Why useful: As a yardstick to compare other algorithms against optimal performance
- No best practical algorithm exists β depends on workload
- Random replacement does pretty badly, though some OS situations use near-random effectively
#2: FIFO (First-In, First-Out)
π Definition β FIFO: Page replacement algorithm that evicts the page that was brought in longest ago.
- Simple to implement: when paging in, put on tail of list; evict from head
- Good: Maybe the oldest page is not being used
- Bad: Have absolutely no information either way
- Performance is typically not good
- Belady's Anomaly: There exist reference strings where the fault rate increases when the process is given more physical memory
#3: Least Recently Used (LRU)
π Definition β LRU (Least Recently Used): Page replacement algorithm that evicts the page that hasn't been used for the longest period of time (past experience as predictor of future behavior).
- Works exceedingly well in general
- Difference from FIFO: LRU looks at past usage (recentness), FIFO looks at arrival time
π Example with reference string: A B C A B D A D B C B (On replacement decisions, LRU would evict the page whose most recent use was furthest in the past)
Implementing LRU
- With time stamps: On every memory reference, time stamp each page; at eviction, scan for oldest
- With stack: Keep a stack of page numbers; on every reference, move the page to top of stack
- Problems: Large page lists, no hardware support for time stamps
Approximating LRU (using PTE reference bit)
Keep a counter for each page. At regular intervals, for each page:
- If ref bit = 0, increment counter (hasn't been used)
- If ref bit = 1, zero counter (has been used)
- Regardless, zero ref bit
The counter contains the number of intervals since the last reference. Page with largest counter is least recently used.
#4: LRU Clock (Not Recently Used / Second Chance)
π Definition β Clock Algorithm (NRU/Second Chance): A page replacement algorithm that arranges physical page frames in a circular list (clock) and uses a "clock hand" to sweep through pages, evicting those with reference bit off.
- Algorithm:
- Sweep through pages in circular order
- If ref bit is off β victim (hasn't been used recently)
- If ref bit is on β turn it off and go to next page
- Arm moves quickly when pages needed; low overhead with plenty of memory
- With large memory, accuracy degrades (add more hands to fix)
Problem with large memory: Solution is to add another clock hand β leading edge clears ref bits, trailing edge (N pages back) evicts pages with ref bit 0.
- Angle too small? Angle too large?
Nth Chance Algorithm
Don't throw page out until the hand has swept by N times. OS keeps a counter per page (number of sweeps). On page fault:
- Reference bit = 1 β clear reference bit, clear counter, go on
- Reference bit = 0 β increment counter; if < N, go on; else replace page
Choosing N: Larger N gives better approximation to LRU; smaller N is more efficient (otherwise may look a long way for free page)
Dirty pages: Take extra overhead to write back to disk when replaced. Common approach:
- Clean pages: N = 1
- Dirty pages: N = 2 (write-back to disk when N=1)
Page replacement: Global or local?
Global replacement: When a process faults and needs a page, take oldest page from the entire system.
- Good: Adaptable memory sharing (e.g., P1 needs 20%, P2 needs 70%)
- Bad: Too adaptable; little protection (what happens to P1 if P2 sequentially reads an array about the size of memory?)
Local replacement: Each process replaces only its own pages.
β Key Takeaways
- Set associative and fully associative caches involve a tradeoff between hardware complexity and miss rate; TLBs are typically fully associative while larger caches use lower associativity. Page replacement algorithms must balance sophistication (for higher hit rates) against speed and hardware support constraints. Demand paging leverages locality to keep memory usage efficient, transparently loading pages only when needed. LRU and its approximations (particularly the Clock algorithm) are the most practical effective policies, while Belady's optimal algorithm serves as an analytical benchmark. Global replacement provides adaptive memory sharing but risks thrashing, motivating local replacement or working set-based approaches.
π§ Quick Revision Questions
- What are the key differences between set associative and fully associative caches, and how does this affect TLB design?
- How does the TLB maintain consistency with page tables during context switches and when page tables change?
- What happens step-by-step during a page fault, from the hardware exception to the OS handler restarting the process?
- Why does FIFO suffer from Belady's Anomaly, and how does LRU avoid this problem?
- How does the Clock (NRU) algorithm approximate LRU, and what modifications (Nth chance, dirty page handling) improve its performance?