π Lecture 35 β Multiprocessors (Cache Coherence Problem)
π Overview: This lecture addresses the critical issue of cache coherence in multiprocessor systems, where shared data replicated across multiple caches can lead to inconsistencies. Understanding this problem is fundamental to designing efficient parallel computing architectures, as it directly impacts data integrity and system performance in both symmetric shared-memory and distributed memory systems.
ποΈ Topics Covered
The lecture begins with a recap of parallel processing architecture, MIMD classification, and the framework for parallel processing. It then introduces the multiprocessor cache sharing concept and the cache coherence problem, explaining formal definitions and features of a coherent system. The main body covers cache coherence on buses, coherence with write-through caches, cache coherence protocols including snooping and directory-based schemes, and a detailed examination of write invalidate versus write broadcast protocols. The lecture concludes with an example snooping protocol using finite state machines for write invalidation and write-back caches.
π Lecture Summary
Recap: Parallel Processing Architecture
Last time, the concept of parallel processing was introduced to improve computer performance by using a collection of processing elements that cooperate and communicate. Flynnβs four categories of computersβSISD (Single Instruction Single Data), SIMD (Single Instruction Multiple Data), MISD (Multiple Instruction Single Data), and MIMD (Multiple Instruction Multiple Data)βform the basis for implementing programming and communication models. The MIMD machines implement parallel processing architecture.
Recap: MIMD Classification
Based on memory organization and interconnect strategy, MIMD machines are classified as Centralized Shared Memory Architecture and Distributed Memory Architecture. In centralized shared memory, subsystems share the same physical centralized memory connected by a bus, characterized by Uniform Memory Access (UMA)βwhere access time to all memory from all processors is the same. Distributed memory consists of individual nodes containing a processor, some memory, I/O, and an interface to an interconnection network, providing more memory bandwidth and lower memory latency.
Recap: Framework for Parallel Processing
The framework defines programming and communication models for centralized shared-memory and distributed memory parallel processing architectures. These models present address space sharing and message passing. The shared-memory communication model has compatibility with SMP hardware and offers ease of programming when communication patterns are complex or vary dynamically. The message-passing communication model has explicit communication that is simple to understand and easier to use for sender-initiated communication.
Multiprocessor Cache Sharing
Today, we examine caching for multi-processing in the symmetric shared-memory architecture, where each processor has the same relationship to the single memory. Small-scale shared-memory machines support caching of both private data and shared data. Private data is used by a single processor, while shared data is replicated in caches of multiple processors for simultaneous use. The program behavior for caching private data is identical to a uniprocessor since no other processor uses the same data.
Multiprocessor Cache Coherence
When shared data are cached, the shared value may be replicated in multiple caches, reducing access latency and fulfilling bandwidth requirements. However, due to differences in communication for load/store and write strategies in caches, values in different caches may not be consistent. This conflict for shared data being read by multiple processors simultaneously is referred to as the cache coherence problem. Informally, a memory system is coherent if any read of a data item returns the most recently written value of that data item.
π Definition β Cache Coherence Problem: The conflict or contention arising when shared data is replicated in multiple caches and values become inconsistent due to different write strategies and communication patterns in multiprocessor systems.
This definition contains two aspects: Coherence, which defines what value can be returned by a read, and Consistency, which determines when a written value will be returned by a read.
Cache Coherency Problem?
Processors P1, P2, and P3 see old values in their caches because there exist several alternatives to write to caches. In write-back caches, the value written back to memory depends on which cache flushes or writes back the value and whenβthe value returned depends on program order, program issue order, or order of completion. The cache coherency problem exists even on uniprocessors where interaction between caches and I/O devices requires infrequent software solutions. However, the problem is performance-critical in multiprocessors where order among multiple processes is crucial and must be treated as a basic hardware design issue.
Order among multiple processes?
Consider a single shared memory with no caches. Every read/write to a location accesses the same physical location, and the operation completes when it does so. This imposes a serial or total order on operations to the location: operations from a given processor are in program order, and the order from different processors is some interleaving that preserves individual program orders. With caches, the "latest" means the most recent in a serial order with operations to a location from a given processor in program order. For the serial order to be consistent, all processors must see writes to the location in the same order.
Formal Definition of Coherence
A memory system is coherent if the results of any execution of a program are such that for each location, it is possible to construct a hypothetical serial order of all operations to the location that is consistent with the execution results. In a coherent system, the operations issued by any particular process occur in the order issued by that process, and the value returned by a read is the value written by the last write to that location in the serial order.
Features of Coherent System
Two features of a coherent system are:
- Write propagation: Value written must become visible to othersβany write must eventually be seen by a read
- Write serialization: Writes to a location are seen in the same order by all
Cache Coherence on buses
Bus transactions and Cache state transitions are fundamentals of uniprocessor systems. Bus transactions pass through three phases: arbitration, command/address, and data transfer. Cache state transition deals with every block as a finite state machine. Write-through, write no-allocate caches have two states: valid and invalid. Write-back caches have one more state: modified (dirty).
Coherence with write-through caches!
The controller snoops on bus events (write transactions) and invalidates/updates cache. In write-through, memory is always up-to-date, so invalidation causes the next read to miss and fetch the new value from memoryβthe bus transaction is indeed write propagation. Bus transactions impose write serialization as writes are seen in the same order.
Cache Coherence Protocols
In a coherent multiprocessor, caches provide both relocation (migration) and replication (duplication) of shared data items. Protocols use different techniques to track sharing status to maintain coherence for multiprocessors, referred to as Cache Coherence Protocols.
Potential HW Coherency Solutions
Two fundamental classes of Coherence protocols are:
- Snooping Protocols: All cache controllers monitor or snoop on the bus to determine whether they have a copy of the block requested on the bus
- Directory-Based Protocols: The sharing status of a block of physical memory is kept in one location, called a directory
Snoopy solutions: Send all requests for data to all processors; processors snoop to see if they have a copy and respond accordingly; requires broadcast since caching information is at processors; works well with bus (natural broadcast medium); dominates for small-scale machines.
Directory-Based schemes: Keep track of what is being shared in one centralized place; distributed memory employs distributed directory for scalability and to avoid bottlenecks; send point-to-point requests to processors via network; scales better than snooping; actually existed before snooping-based schemes.
Basic Snooping Protocols
There are two ways to maintain coherence requirements using snooping protocols: 1. Write Invalidate Method: Ensures that processor has exclusive access to the data item before it writes that item, and all other cached copies are invalidated or canceled on write. Exclusive access ensures no other readable or writable copies of an item exist when the write occurs.
Write Invalidate Protocol uses multiple readers and single writer. For write to shared data, an invalidate information is sent to all caches, and the controller snoops and invalidates any copies. For read miss in write-through, memory is always up-to-date; in write-back, it snoops in caches to find the most recent copy.
π Example: Invalidation protocol for snooping bus with write-back cache assuming both CPU A and B caches do not initially hold X, and memory value of X is 0. When CPU A writes 1 to X, exclusive access is required, and any copy held by the reading processor must be invalidated. When B reads, it misses in cache and is forced to get a new copy of data. The exclusive write access prevents any other processor from writing simultaneously. When 2nd miss by B occurs, CPU A responds with the value, canceling memory response. Both Bβs cache and memory contents of X are updated. The invalidation for memory location X occurs when A attempts to write 1.
2. Write Broadcast Protocol: Instead of invalidating, this protocol updates all cached copies of a data item when that item is written. It is particularly used for write-through caches. For write to shared data, processors snoop and update any copies by broadcasting on the bus.
π Example: Write update protocol for snooping bus with write-back cache assuming both CPU A and B caches do not initially hold X, and memory value is 0. When CPU A writes a 1 to memory X, it updates the value in caches of A and B and the memory.
Write Invalidate versus Broadcast
Invalidate requires one transaction for multiple writes to the same word. Invalidate uses spatial locality: one transaction for write to different words in the same block. Broadcast has lower latency between write and read.
An Example Snooping Protocol
A bus-based protocol is implemented by incorporating a finite state machine controller in each node. This controller responds to requests from the processor and from the bus based on the type of request, whether it is hit or miss in the cache, and the state of the cache block specified in the request.
Each block of memory is in one of three states: (Shared) Clean in all caches and up-to-date in memory, OR (Exclusive) Dirty in exactly one cache, OR Not in any caches. Each cache block is in one of three states: Shared (block can be read), OR Exclusive (cache has only copy, writable, and dirty), OR Invalid (block contains no data). Read misses cause all caches to snoop bus. Writes to clean line are treated as misses.
Finite State Machine for Write Invalidation Protocol and Write Back Caches
The state machine has three states: Invalid, Shared (read only), and Exclusive (read/write). Cache states are shown in circles where access permitted by CPU without a state transition is shown in parenthesis. The stimulus causing state transition is shown on the transition arc in yellow, and the bus action generated is shown in orange. The state in each cache node represents the state of the selected cache block specified by the processor or bus request. For simplicity, states of the protocol are duplicated to represent transitions based on CPU request and transitions based on bus request.
Snoopy-Cache State Machine-I (CPU requests): A read miss in exclusive or shared state and a write miss in the exclusive state occurs when the address requested by CPU does not match the address in the cache block. An attempt to write a block in shared state always generates a miss, even if the block is present in cache, since the block must be made exclusive. In read hit, shared and exclusive states read data in cache and address the conflict miss. The invalid state places read miss on the bus. For write hit, exclusive state writes data in cache, and shared state places write miss on bus. In write miss, invalid state places miss on bus; shared and exclusive states address conflict miss; shared state places write miss on bus, while exclusive state writes back block and then places write miss on bus.
Snoopy-Cache State Machine-II (Bus requests): Whenever a bus transaction occurs, all caches containing the cache block specified in the bus transaction take action according to this state machine. Memory provides data on a read miss for a block that is clean in all caches. For read miss, the shared state takes no action and allows memory to service the read miss; the exclusive state attempts to share the data, places the cache block on the bus, and changes state to shared. For write miss, the shared state attempts to write a shared block and invalidates the block. The exclusive state attempts to write a block that is exclusive elsewhere; writes back the cache block and makes the state invalid.
β Key Takeaways
The cache coherence problem is a fundamental challenge in multiprocessor systems where shared data replicated in multiple caches can become inconsistent. The two essential features of a coherent system are write propagation (writes become visible to all) and write serialization (writes to a location are seen in the same order by all). Two main classes of coherence protocols exist: snooping protocols (where cache controllers monitor the bus) and directory-based protocols (where sharing status is kept in a centralized directory). The two basic snooping techniques are write invalidate (which grants exclusive access before writing and invalidates other copies) and write broadcast (which updates all cached copies on writes). Finite state machines with three states (Invalid, Shared, Exclusive) provide the hardware implementation mechanism for snooping protocols, with separate state transitions for CPU requests and bus requests.
π§ Quick Revision Questions
- What are the two fundamental aspects of memory behavior in a coherent system, and how do they differ?
- Explain the difference between write invalidate and write broadcast protocols, and give one advantage of each.
- What are the three states in the finite state machine for write invalidation protocol with write-back caches?
- How does a snooping protocol handle a write to a shared block when using write invalidation?
- Why is write serialization important in a coherent multiprocessor system?
π Lecture 36 β Multiprocessors (Cache Coherence Problem β¦ Contβd)
π Overview: This lecture continues the discussion on cache coherence in symmetric shared-memory multiprocessors, focusing on the practical implementation of snooping protocols using finite state machine controllers. It then introduces coherence solutions for distributed memory architectures, specifically directory-based protocols, and compares their performance and complexity with snooping approaches.
ποΈ Topics Covered
The lecture begins with a detailed recap of the cache coherence problem, write propagation, and serialization, followed by a step-by-step example of an invalidation-based snooping protocol. It then covers implementation complications like write races and snooping cache conflicts, explores variations like MESI, Berkeley, and Illinois protocols, and concludes with a thorough explanation of directory-based protocols for distributed shared memory systems, including state transition diagrams and message types.
π Lecture Summary
Recap: Cache Coherence Problem
Last time, we discussed the sharing of caches for multi-processing in the symmetric shared-memory architecture, where each processor has the same relationship to the single memory. We distinguished between private data (used by a single processor) and shared data (replicated in caches of multiple processors). The cache coherence problem results from inconsistency in caching shared data being read simultaneously. Using write-back caches (where values written back depend on which cache flushes), we saw that the coherency problem exists even on uniprocessors due to I/O interaction, but in multiprocessors it is performance-critical.
Recap: Order among multiple processes
For a single shared memory without caches, a serial or total order is imposed on operations. With caches, the serial order must be consistent β all processors must see writes to a location in the same order. In a coherent system: operations from any process occur in the order issued, and a read returns the value written by the last write in the serial order. Two features of a coherent system are write propagation and write serialization.
Recap: Multiprocessor cache Coherence & Coherency Solutions
To implement cache coherence, multiprocessors extend both bus transactions and state transitions. The cache controller snoops on bus events (write transactions) and invalidates/updates cache. Two fundamental classes of coherence protocols are:
- Snooping Protocols: All cache controllers monitor (snoop) the bus to detect if they have a copy of the requested block.
- Directory-Based Protocols: The sharing status of a block is kept in one location called a directory.
Recap: Basic Snooping Protocols
Snooping protocols use two techniques: write invalidate (processor gets exclusive access before writing, invalidating all other copies) and write broadcast (all cached copies are updated). Invalidate uses spatial locality (one transaction for multiple writes to the same block), while broadcast has lower latency between write and read. The finite state machine controller implements snooping protocols, responding to processor and bus requests based on request type, hit/miss status, and cache block state (Shared, Exclusive, or Invalid).
Example: Working of Finite State Machine Controller
Today we continue with an example assuming two processors P1 and P2, each with its own cache, sharing memory on a bus. The table shows processor status, bus transactions, and memory. Initially, cache state is Invalid (block not in cache); memory blocks A1 and A2 map to the same cache block (A1 β A2).
π Definition β Write-Back Cache: A cache where write operations update the cache block, and the updated value is only written to memory when the block is evicted.
Step 1 β P1 writes 10 to A1: A write miss on the bus occurs; the state transitions from Invalid β Exclusive. Step 2 β P1 reads A1: CPU read HIT occurs; the FSM stays in Exclusive state. Step 3 β P2 reads A1: P2 (in Invalid) has a read miss; its state changes from Invalid β Shared. P1 (in Exclusive) sees a remote read write-back, its state changes from Exclusive β Shared. The value (10) is read from shared memory into both caches at A1. Step 4 β P2 writes 20 to A2: P1 sees a remote write, state changes from Shared β Invalid. P2 sees a CPU write, places write miss on bus, state changes from Shared β Exclusive, and writes value 20 to A1. Step 5 β P2 writes 40 to A2: P2 (in Exclusive) has a CPU write miss and initiates write-back to P2 at A2, remaining in Exclusive state with address A2 and value 40.
π Formula: State Transition Logic β The cache controller transitions states based on: (CPU request or Bus snoop) AND (Hit/Miss status).
π Example: In the step-by-step sequence, when P2 writes 20 to A2, P1βs cache is invalidated because it held a shared copy of A1 (same cache block mapping). This demonstrates the write invalidate protocol in action.
Implementation Complications
While the FSM works well, these complications arise:
- Write Races: Occur when one processor wants to update a cache block but another gets the bus first and writes the same block. The bus transaction is a two-step process: arbitrate for bus, then place miss and complete. If a miss occurs while waiting for bus, the processor must handle the miss (invalidate) and restart. Split transaction buses allow multiple outstanding transactions for a block, preventing races but requiring tracking of multiple misses for one block.
Snooping Cache Conflict
The CPU accesses cache while bus transactions check cache tags. Since every bus transaction checks tags, interference with CPU can occur. Two methods to reduce interference:
- Duplicate set of tags for L1 caches: CPU uses one set, snoop uses another. The CPU stalls only when snoop detects a copy and tags need updating.
- Multi-level caches with inclusion: L2 cache duplicates L1 content, provided L2 obeys inclusion (L1 content is in L2). CPU activity goes to L1, snoop activity to L2. If snoop hits, it arbitrates L1 to update/get data, stalling the CPU. This can be combined with duplicate tags.
Snooping Cache Variations: MESI Protocol
The MESI Protocol has four states: Modified, Exclusive, Shared, Invalid. Here, Exclusive means exclusively cached but clean upon loading (memory up-to-date). Bus serializes writes; getting the bus ensures no one else can perform memory operations.
Snooping Cache Variations: Berkeley Protocol
The Berkeley Protocol allows cache-to-cache transfers on the shared bus. It adds the notion of owner β the cache that has the block in a Dirty state is the owner (the last one who wrote). The owner is responsible to transfer data on reads and update main memory. If no cache owns the block, memory is the owner.
Summary Snooping Cache Variations
- Illinois Protocol: States are Private Dirty, Private Clean, Shared, Invalid.
- MESI Protocol: Modified (private, memory not up-to-date), eXclusive (private, memory up-to-date), Shared (shared, memory up-to-date), Invalid.
- If read sourced from memory β Private Clean; if sourced from another cache β Shared. Can write in cache if held Private Clean or Private Dirty.
Snoop Cache Extensions
Extensions include:
- Berkeley: Fourth state Ownership; Shared β Modified needs only invalidate (upgrade request, no memory read).
- MESI: Clean exclusive state (no miss for private data on write).
- Illinois: Cache supplies data when in Shared state (no memory access).
π Definition β Upgrade Request: A bus transaction that requests permission to write a shared block without fetching the data from memory, as the block already exists in the requesting cache.
Larger Microprocessors & Directory Based Protocol
Larger systems use separate memory per processor (distributed memory) with local/remote access via a memory controller. One coherency solution uses non-cached pages; the alternative is a directory containing information for every block in memory, tracking state and which caches have copies. Using information per memory block (vs. per cache block):
- PLUS: Simpler protocol (centralized location).
- MINUS: Directory size is a function of memory size (vs. cache size for simpler protocols).
Directory Based Protocol: Distributed Shared Memory
The directory-based protocol is similar to snoopy protocols but uses the directory. Three states:
- Shared: β₯1 processor has data, memory up-to-date.
- Uncached: No processor has it; not valid in any cache.
- Exclusive: 1 processor (owner) has data; memory out-of-date. In addition, must track which processors have data when in Shared state (usually using a bit vector). Key assumptions: writes to non-exclusive data cause a write miss; processor blocks until access completes; messages are received and acted upon in order sent. Three processor nodes involved: Local node (request originates), Home node (memory location resides), Remote node (has a copy of cache block).
π Example: Message types include: Read miss (P, A) β request data; Write miss (P, A) β request exclusive ownership; Invalidate (A) β invalidate shared copies; Fetch (A) β fetch block to home directory; Data value reply (Data) β return data as read miss response; Data write-back (A, Data) β write-back data as invalidate response.
State Transition Diagram for an Individual Cache Block in a Directory Based System
States are identical to the snoopy case. Transactions are caused by read misses, write misses, invalidates, and data fetch requests. Write misses that were broadcast on the bus for snooping now result in explicit invalidate and data fetch requests. On a write, the full cache block must be read.
State Transition Diagram for the Directory
The directory uses the same states and structure. Two actions: update directory state and send messages to satisfy requests. The controller tracks all copies of a memory block and updates the sharing set (Sharers) while sending messages.
Example Directory Protocol
Messages sent to the directory cause: update the directory and send more messages to satisfy the request.
- Block in Uncached state (memory has current value):
- Read miss: Requesting processor gets data from memory; becomes only sharing node; state becomes Shared.
- Write miss: Requesting processor gets value and becomes the sharing node; state becomes Exclusive; Sharers indicates owner.
- Block in Shared state (memory up-to-date):
- Read miss: Requesting processor gets data from memory; added to sharing set.
- Write miss: Requesting processor gets value; all processors in Sharers get invalidate messages; Sharers set to requesting processor; state becomes Exclusive.
- Block in Exclusive state (value in owner's cache, identified by Sharers):
- Read miss: Owner processor sent a data fetch message; owner's block state transitions to Shared; owner sends data to directory, which is written to memory and sent to requestor; requesting processor added to Sharers (owner still has readable copy); state becomes Shared.
- Data write-back: Owner is replacing the block, writes it back; memory copy becomes up-to-date; block becomes Uncached; Sharers becomes empty.
- Write miss: A message sent to old owner causes it to send the block value to directory, which sends it to requesting processor (new owner); Sharers set to new owner; state becomes Exclusive.
π‘ Why this matters: Directory-based protocols scale to larger systems than snooping protocols because they avoid broadcasting on a shared bus. The directory acts as a centralized coherency point, eliminating bus arbitration bottlenecks and enabling distributed shared memory architectures.
Summary
- Caches contain all information on state of cached memory blocks.
- Snooping and Directory Protocols are similar in logic.
- The bus makes snooping easier because of broadcast capability.
- Directory has an extra data structure to keep track of the state of all cache blocks.
β Key Takeaways
The write invalidate snooping protocol using a finite state machine with Invalid/Shared/Exclusive states is the foundational coherence mechanism, and its step-by-step operation with two processors demonstrates how cache states transition during reads, writes, and remote accesses. Implementation complications like write races and snooping cache conflicts require solutions such as split transaction buses, duplicate tags, and multi-level caches with inclusion. Protocol variations (MESI, Berkeley, Illinois) each add optimizations: MESI's clean exclusive avoids write misses for private data, Berkeley adds ownership for cache-to-cache transfers, and Illinois allows cache supply on shared reads. For larger systems, directory-based protocols scale better than snooping by avoiding broadcast, using a directory to track which processors share each block and sending explicit invalidate/fetch messages only to affected nodes, though at the cost of extra storage for directory entries.
π§ Quick Revision Questions
- In the example FSM protocol, what state transition occurs for P1 when P2 reads A1 from Exclusive state?
- What two methods reduce interference between CPU cache access and snooping bus transactions?
- What is the key difference between the MESI protocol's 'Exclusive' state and the basic three-state protocol's 'Exclusive' state?
- In a directory-based protocol, what messages are sent to the old owner when a write miss occurs on a block in Exclusive state?
- What are the two main actions performed by the directory state machine when it receives a message?
π Lecture 37 β Multiprocessors (Performance and Synchronization)
π Overview: This lecture examines the performance of multiprocessors using symmetric shared-memory and distributed shared-memory architectures. It then introduces hardware synchronization primitives and techniques essential for coordinating access to shared data in parallel systems.
ποΈ Topics Covered
The lecture covers performance analysis of multiprocessors including coherence misses like true-sharing and false-sharing, performance factors for both bus-based and directory-based systems. It then explores synchronization mechanisms, focusing on hardware primitives such as atomic exchange, test-and-set, and the load-linked/store-conditional instruction pair, with examples of building locks and atomic operations.
π Lecture Summary
Recap: Cache Coherence Problem
The lecture begins by recapping that cache coherence is a performance-critical problem in both symmetric shared-memory and distributed shared-memory architectures. The protocols used to maintain coherence include snooping protocols (using write invalidate and write broadcast) and directory-based protocols. Snooping protocols use a three-state FSM (Shared, Exclusive, Invalid) with variations like MESI (Modify, Exclusive, Shared, Invalid), Barkley (Owned-Exclusive, Owned-Shared, Shared, Invalid), and Illinois (Private Dirty, Private Clean, Shared, Invalid). Directory-based protocols are used in larger systems with distributed memory, tracking the state of every block in every cache via a directory.
Example: Working of Finite State Machine Controller
A detailed example with two processors P1 and P2, each with its own cache, memory, and directory, illustrates directory-based protocol operation. The FSM transitions from the Uncached state (data in memory only) based on messages like read miss, write miss, invalidates, and data fetch requests.
A walkthrough assumes A1 and A2 map to the same cache block:
- Step 1: P1 writes 10 to A1. A write miss occurs. The data value reply message is sent to the controller. P1 is inserted in the directory sharer-set
{P1}. The state transitions from Uncached to Exclusive. - Step 2: P1 reads A1. A CPU read hit occurs, so the FSM stays in Exclusive state.
- Step 3: P2 reads A1. A read miss occurs. P1, being in Exclusive state, asserts remote read write-back and changes to Shared. P2 changes from Uncached to Shared. The value 10 is read into both caches, and the sharer-set becomes
{P1, P2}. - Step 4: P2 writes 20 to A2. Since A1 and A2 map to the same block, P1 sees a remote write and changes from Shared to Invalid. P2 places a write miss on the bus and changes from Shared to Exclusive, writing value 20 to A1. The directory for A1 now has sharer-set
{P2}. - Step 5: P2 writes 40 to A2. P2, in Exclusive state, experiences a write miss at A2. The directory for A2 is in Exclusive state with P2 in the sharer-set. P2 write-backs 20 at A1, making the directory for A1 Uncached with an empty sharer-set and value 20 in memory. P2 remains in Exclusive state with address A2 and value 40.
π Definition β True Sharing Miss: A miss that arises from the communication of data through the cache-coherence mechanism, where data is actually shared between processors. π Definition β False Sharing Miss: A miss that occurs when a block is invalidated and a subsequent reference causes a miss, even though the word being written and the word being read are different. The miss occurs because of the single valid bit per cache block, not because of actual data sharing.
π Example of True and False Sharing: Consider words A1 and A2 in the same cache block, in Shared state in caches of P1 and P2.
| Time | P1 | P2 |
|---|---|---|
| 1 | Write A1 | |
| 2 | Read A2 | |
| 3 | Write A1 | |
| 4 | Read A2 | |
| 5 | Write A2 |
- Event 1: P1 Write A1 β True sharing miss (A1 was read by P2 and must be invalidated from P2).
- Event 2: P2 Read A2 β False sharing miss (A2 was invalidated by P1's write to A1, but the value of A1 is not used in P2).
- Event 3: P1 Write A1 β False sharing miss (block is marked shared due to P2's read, but P2 did not read A1).
- Event 4: P2 Write A2 β False sharing miss (block is marked shared due to P2's read in event 2, but P2 did not write A2).
- Event 5: P1 Read A2 β True sharing miss (the value being read by P2 was written by P2 in event 4).
Performance of Multiprocessors: Symmetric Shared-Memory Architecture
In bus-based multiprocessors using invalidation protocols, overall cache performance is a combination of uniprocessor cache miss-traffic and traffic caused by communication (invalidation and subsequent cache misses). Changing processor count, cache size, and block size affects these components. Coherence misses arise from inter-processor communication and come from two sources:
- True Sharing: Arises from actual communication of data. The first write by a processor to a shared cache block causes an invalidation to establish ownership. When another processor reads the modified word, a miss occurs and the block is transferred.
- False Sharing: Arises from the use of invalidation-based coherence algorithms with a single valid bit per cache block. A block is invalidated even when the word being written and the word being read are different, causing an extra cache miss without actual data communication.
π‘ Why this matters: Distinguishing true from false sharing is critical for optimizing parallel programs. False sharing can be reduced by increasing block size or restructuring data layout, while true sharing is inherent to the algorithm's communication pattern.
Performance of Multiprocessors: Distributed Shared-Memory Architecture
The performance of directory-based multiprocessors depends on factors like processor count, cache size, and block size. Additionally, the location of requested data β dependent on initial allocation and sharing pattern β significantly influences performance. The distribution of memory requests between local memory and remote memory is key, as it affects global bandwidth consumption and request latency. Graphs show that as cache size grows, miss rates decrease; local miss rates decline steadily, while the decline in remote miss rates depends on coherence misses. Similarly, increasing block size reduces miss rates.
Synchronization
Synchronization is needed to ensure it is safe for different processes to use shared data. Mechanisms are built with user-level software routines that rely on hardware-supplied synchronization instructions. For small multiprocessors, uninterruptable instructions that fetch and update memory atomically are used (atomic operations). For large-scale multiprocessors, synchronization can be a bottleneck, and techniques are needed to reduce contention and latency.
π Definition β Atomic Operation: An operation that reads and updates a memory value in a single, indivisible step, preventing other processors from interfering.
Hardware Primitives: Uninterruptable Instructions
The basic requirement for implementing synchronization is a set of hardware primitives with the ability to atomically read and modify a memory location. One typical operation is Atomic exchange, which interchanges a value in a register for a value in memory. Other primitives include Test-and-Set (tests a value and sets it if the test passes) and Fetch-and-Increment (returns a memory value and atomically increments it).
- Atomic Exchange for a Simple Lock: A lock where 0 indicates free and 1 indicates unavailable. A processor exchanges 1 (in a register) with the memory address of the lock. If the returned value is 1, another processor has claimed access; if 0, the processor acquires the lock (and the memory value is changed to 1, preventing other exchanges from retrieving 0).
π Example of Simultaneous Exchange: If two processors try to exchange simultaneously, the race is broken when one exchanges first and returns 0, and the second returns 1.
Because implementing a single atomic instruction in hardware is complex, modern multiprocessors use a pair of instructions: Load Linked (LL) and Store Conditional (SC).
- Load Linked (LL): Returns the initial value from a memory location.
- Store Conditional (SC): Returns 1 if it succeeds (no other store to the same memory location since the preceding LL) and 0 otherwise.
If the contents of the memory location specified by LL are changed before the SC to the same address occurs, the SC fails.
π Example: Implementing Atomic Exchange with LL & SC
try: MOV R3,R4 ; mov exchange value
ll R2,0(R1) ; load linked
sc R3,0(R1) ; store conditional
beqz R3,try ; branch if store fails (R3 = 0)
mov R4,R2 ; put load value in R4
At the end, R4 and the memory location specified by R1 have been atomically exchanged.
π Example: Implementing Atomic Fetch & Increment with LL & SC
try: ll R2,0(R1) ; load linked
addi R2,R2,#1 ; increment (OK if regβreg)
sc R2,0(R1) ; store conditional
beqz R2,try ; branch if store fails (R2 = 0)
Since SC only checks that its address matches the link register, register-register instructions can safely be placed after LL, but the number of instructions between LL and SC must be kept small.
Summary
The lecture concludes the series on multiprocessors by stating that multiprocessors are highly effective for multi-programmed workloads and commercial workloads like web searching. The centralized memory architecture (SMPs) maintains a single centralized memory with uniform access time, while distributed shared-memory multiprocessors (DSMs) have non-uniform memory architecture and achieve greater scalability. These advantages can be partially combined in architectures like Sun Microsystems' Wildfire, where large SMPs (e.g., E6000) are used as nodes to maximize uniform memory access, and scalability is achieved via the Wildfire Interface (WFI), which can connect 2 or 4 E6000 multiprocessors.
β Key Takeaways
The key performance issue in multiprocessors is coherence misses, which are classified as true-sharing misses (arising from actual data communication) and false-sharing misses (arising from the block-based invalidation mechanism when different words within a block are accessed by different processors). Performance in distributed shared-memory architectures is additionally influenced by the location of data, making the distribution of local versus remote memory requests critical. Synchronization is implemented using hardware primitives that allow atomic read-modify-write operations, with modern systems typically using the Load Linked/Store Conditional instruction pair to build locks and atomic operations like exchange and fetch-and-increment. The FSM controller for directory-based protocols tracks block states (Uncached, Shared, Exclusive) and uses messages between local, home, and remote nodes to maintain cache coherence.
π§ Quick Revision Questions
- What is the difference between a true-sharing miss and a false-sharing miss? Provide a specific example with two processors and two addresses in the same cache block.
- In the FSM controller example, what happens when P2 writes to address A2 while P1 has A1 (same cache block) in Shared state? Describe the state transitions for both processors and the directory.
- How does the Load Linked/Store Conditional (LL/SC) pair implement an atomic operation? Write the assembly code sequence for an atomic exchange using LL/SC.
- Why is the distribution of memory requests between local and remote memory critical for performance in distributed shared-memory architectures?
- Explain how a simple lock works using the atomic exchange primitive. What does it mean if the exchange returns a value of 1 versus 0?
π Lecture 38 β Input Output Systems (Storage and I/O Systems)
π Overview: This lecture transitions from parallel processing architectures to the critical role of I/O systems in overall computer performance. It introduces storage technologies, focusing on magnetic disks, and explains how neglecting I/O can severely limit system speed-up according to Amdahl's Law. The lecture also covers I/O performance metrics and processor-interface control structures.
ποΈ Topics Covered
A recap of multiprocessing architectures (SIMD, MISD, MIMD, cache coherence) is followed by a discussion of why I/O systems are now the dominant performance bottleneck. The lecture then introduces magnetic disk storage technology, its historical evolution from 1956 to the 1990s, and alternative storage technologies like tape and optical disk. Key I/O performance parameters (diversity, capacity, latency, bandwidth) and the producer-server model are explained, along with processor interface issues including isolated I/O, memory-mapped I/O, programmed I/O, interrupts, DMA, and I/O processors.
π Lecture Summary
Recap: Multiprocessing & The Outside Processor
The lecture begins by recapping that Parallel Architecture uses cooperating processing elements to solve large problems fast. This includes SIMD, MISD, and MIMD machines, where MIMD facilitates complete parallel processing. MIMD is classified into Centralized Shared Memory Architecture (uniform access time) and Distributed Memory Architecture (non-uniform memory, greater scalability). The cache coherence problem is solved via Snooping algorithms (for centralized) and Directory Based Protocols (for distributed).
The crucial transition is that while processing power and memory size double every 18 months, disk positioning rate (seek + rotate) doubles only every 10 years. This creates a massive performance gap, making I/O the dominant bottleneck.
π‘ Why this matters: The lecture establishes that a fast processor is useless without a fast I/O system to feed it data.
Introduction: Outside the Processor
The overall computer performance is measured by throughput, which is heavily influenced by I/O. Neglecting I/O is like a car with a powerful engine but no wheels. Using Amdahl's Law, the lecture shows the impact: If CPU time is sped up 10x but I/O is ignored, the overall speedup is only 5 (a 50% loss). If CPU is sped up 100x, the overall speedup is only 10 (a 90% loss). Therefore, I/O performance increasingly limits the system.
An I/O System comprises Storage I/Os (secondary/tertiary storage like magnetic disk, tape, CD) and Communication I/Os (I/O Bus system interconnecting processor/memory with devices).
Disk Storages: Technology Trends & Historical Perspective
Disk capacity has improved dramatically. Before 1990, it doubled every 36 months; now it doubles every 18 months. This is driven by computing paradigm shifts from batch to on-line processing (1950s) to ubiquitous computing (1990s) in phones, books, and cars. This motivated smaller, cheaper embedded storage and high-capacity data utilities.
The historical development of magnetic disks:
- 1956-1970s: IBM Ramac and Winchester for mainframes; form factor shrinking from 27β to 14β.
- 1970s: 5.25β floppy disk; early industry standard interfaces (ST506, SASI, SMD, ESDI).
- Early 1980s: Era of PCs; end of proprietary interfaces.
- Mid 1980s: Client/server computing; accelerated disk downsizing to 5.25β and 3.5β; industry standards like SCSI, IPI, IDE.
- Late 1980s-1990s: Era of laptops; 2.5β and 1.8β form factors. DRAM and flash RAM challenged disks but were still expensive. Optical disks (CD-ROM) found a niche despite poor performance.
A key trend shows the DRAM to Disk ratio: It peaked at ~40% in 1986 but fell to ~15% by 1998.
Devices: Magnetic Disks
- Purpose: Long-term, nonvolatile, large, inexpensive, slow level in the storage hierarchy.
- Characteristics: Seek Time (~8 ms avg) β positional latency; Rotational Latency (half a revolution); Transfer rate (~a sector per ms, 5-15 MB/s).
- Capacity: Gigabytes, quadruples every 3 years.
- Speed Example: 7200 RPM = 120 RPS β 8 ms per revolution β avg rotational latency = 4 ms. 128 sectors/track β 0.25 ms per sector. 1 KB per sector β 16 MB/s.
I/O Performance Parameters
I/O performance has unique parameters with no CPU counterpart:
- Diversity: Which I/O devices can connect to the CPU.
- Capacity: How many I/O devices can connect.
- Latency: Overall response time to complete a task.
- Bandwidth: Number of tasks completed in a specified time (throughput).
An I/O system is in equilibrium when the rate of arriving I/O requests equals the rate of departing requests after being serviced. The producer-server model (a FIFO queue with a server) is used: Response Time = Queue Time + Device Service Time.
π Formula: Response Time = Time to Queue + Device Service Time β The total time from when a task arrives in the I/O buffer to when the server finishes it.
π Example: A graph shows that minimum response time is achieved at only 10% throughput, while achieving 100% throughput takes 7-8 times the minimum response time. This illustrates the tradeoff between response time and throughput.
I/O Transaction Time
The transaction time of a computer is the sum of three times:
- Entry Time: Time for user to enter a command (avg 0.25 sec; from keyboard 4.0 sec).
- System Response Time: Time between command entry and system response.
- Think Time: Time from reception of response until user enters next command.
π Example: What happens if system response time shrinks from 1.0 sec to 0.3 sec?
- With Keyboard (entry 4.0 sec, think 9.4 sec): Shaving off 0.7 sec from response saves 4.9 sec (34%).
- With Graphics (entry 0.25 sec, think 1.6 sec): Shaving off 0.7 sec saves 2.0 sec (70%). This shows that faster response time leads to greater productivity.
Processor Interface Issues
-
Processor Interface Types:
- Isolated I/O: Uses a separate I/O bus and requires special
in/outinstructions. - Memory Mapped I/O: I/O devices appear as memory locations; uses standard
load/storeinstructions. - Interrupts: Devices signal the processor when ready.
- Isolated I/O: Uses a separate I/O bus and requires special
-
I/O Control Structures:
- Polling (Programmed I/O): CPU continuously checks device status; inefficient.
- Interrupt Driven I/O: CPU continues working; device interrupts CPU when ready.
- Direct Memory Access (DMA): A DMAC (DMA Controller) handles data transfer between device and memory. CPU sends starting address, direction, and length, then issues "start". DMAC provides handshake signals.
- I/O Processors (IOP): More intelligent than DMAC. The CPU issues an instruction to the IOP, the IOP looks in memory for commands, directly controls device-to-memory transfers, and interrupts CPU when done.
β Key Takeaways
The most critical point is that I/O systems, not just the CPU, are the primary bottleneck to overall computer performance, as dramatically illustrated by Amdahl's Law where a 100x CPU speedup is reduced to 10x by a slow I/O system. Students must understand the tradeoff between response time and throughput in an I/O system, and know the three components of I/O transaction time: entry, system response, and think time. Magnetic disk performance is governed by seek time, rotational latency, and transfer rate. Finally, the different processor interfaces (isolated vs. memory-mapped) and control structures (polling, interrupt, DMA, IOP) represent a hierarchy of efficiency for managing I/O operations.
π§ Quick Revision Questions
- According to Amdahl's Law, if CPU time is sped up 100 times but I/O time is not improved, what is the overall system speedup?
- List the three components of a disk's I/O transaction time.
- What are the four main I/O control structures, and which one offloads the most work from the CPU?
- What are the two key performance parameters for an I/O system that have no direct counterpart in CPU performance metrics?
- In the producer-server model for I/O, what is the formula for a task's total response time?