π Lecture 45 β Putting It All Together (Review: Lecture 1 - 43)
π Overview: This lecture provides a comprehensive review of the entire Advanced Computer Architecture course, covering all nine modules from Introduction and Quantitative Principles through Networks and Clusters. It systematically revisits the key concepts, definitions, formulas, and techniques studied across all lectures, serving as a final consolidation for exam preparation.
ποΈ Topics Covered
The lecture reviews all nine course modules in sequence: Introduction and Quantitative Principles including price-performance design and Amdahl's Law; Instruction Set Architecture covering ISA taxonomy, operand types, and addressing modes; Computer Hardware Design including datapath implementations, single/multiple cycle approaches, and pipelining with hazards; Dynamic ILP techniques including scoreboarding, Tomasulo's algorithm, branch prediction, and speculation; Static ILP approaches including superscalar, VLIW, and vector processors along with compiler scheduling techniques; Memory Hierarchy System covering caching principles and locality; Multiprocessing including parallel architectures and cache coherence; I/O Systems covering interconnects, bus protocols, and arbitration schemes; and Networks and Clusters including interconnection topologies and internetworking.
π Lecture Summary
Module 1: Introduction and Quantitative Principles
The course began by distinguishing computer organization from computer architecture. Architecture refers to attributes visible to the programmer or compiler writer such as instruction set, memory addressing, and I/O mechanisms. Organization refers to how features are implemented, for example, control signals generated using FSM or microprogramming principles. The architecture of processor family members is identical, whereas organization may differ between family members.
Computer development began academically in 1944-49 when John von Neumann introduced the stored-program computer concept called EDVAC (Electronic Discrete Variable Automatic Computer). Commercially, the first machine was built by Eckert-Mauchly Computer Corporation in 1949. In 1971, Intel introduced the first cheap microprocessor, the 4004, followed by the 80x86 series. By 1998, over 350 million microprocessors were in use, rising to over a billion by 2006. Technological developments from vacuum tubes to VLSI circuits produced four computer generations. The course viewed Computer Architecture from four perspectives: Processor Design, Memory Hierarchy, Input/output and Storage, and Multiprocessor and Network Interconnection.
π Definition β Price-performance design: The balancing act where high-performance designers may not prioritize cost, while low-cost designers may sacrifice performance, lying between these extremes to balance cost versus performance.
π Definition β Benchmarks: Programs specifically chosen to measure performance; five levels include Real Applications (scientific programs), Modified Applications (real programs with modified blocks), Kernels (small key pieces from real programs), Toy Benchmarks (small codes for beginners), and Synthetic Benchmarks (artificially created programs).
π Amdahl's Law: Speedup (E) = Execution Time without Enhancement / Execution Time with Enhancement = Performance with Enhancement / Performance without Enhancement. This defines speedup due to enhancement E that accelerates a fraction F of the task. π‘ Why this matters: Amdahl's Law quantifies that system speedup is limited by the slowest part of the system, making it fundamental for performance analysis.
Module 2: Instruction Set Architecture
The three pillars of computer architecture are hardware, instruction set, and software. Hardware facilitates software execution, and the instruction set serves as the interface between hardware and software. Focus areas included ISA Taxonomy, operand types, operation types, and memory addressing modes.
π Definition β ISA Taxonomy: Classification of instruction set architectures including Stack Architecture, Accumulator Architecture, and General Purpose Register Architecture (further divided into Register-Memory, Register-Register/Load-Store, and Memory-Memory, which is obsolete).
Operand Types: Integer, FP (Floating Point), and Character. Operand Sizes: Half word, word, double word. Classification of operations: Arithmetic, data transfer, control, and support operations.
Operand Addressing Modes: Immediate, register, direct (absolute), and Indirect. Classification of Indirect Addressing: Register, indexed, relative (with displacement), and memory. Special Addressing Modes: Auto-increment, auto-decrement, and scaled. Control Instruction Addressing Modes: Branch, jump, and procedure call/return.
Module 3: Computer Hardware Design
At a higher level, the CPU consists of two sub-systems: Datapath (the path facilitating information transfer between registers/memory/I/O) and Control (hardware generating signals to control step sequences and direct information flow through the datapath).
π Definition β Datapath: The arithmetic organ of Von Neumann's stored-program organization, typically implemented as Unibus structure, 2-bus structure, or 3-bus structure, based on single cycle, multiple cycle, or pipelined architecture concepts. It consists of registers, internal buses, arithmetic units, and shifters.
Each register in the register file has: a load control line enabling data load, a set of tri-state buffers between output and bus, and a read control line enabling buffer and placing register on bus.
Single Cycle vs Multiple Cycle: In Single Cycle implementation, cycle time accommodates the longest instruction (Load instruction). In Multiple Cycles implementation, cycle time accommodates the longest step (memory read/write). Consequently, Single Cycle cycle time can be five times longer than Multiple Cycle implementation.
Pipelining is a fundamental concept where instructions complete in multiple steps using distinct resources, starting the next instruction while working on the current one. Pipeline Hazards include Structural hazards (same resource accessed by multiple instructions, removed by multiple resources or inserting stalls), Data hazards (attempt to read invalid data, removed by stalls and forwarding), and Control hazards (attempt to branch before condition evaluation).
Four ways to handle control hazards: 1) Stall until branch direction is clear, 2) Predict Branch Not Taken, 3) Execute successor instructions in sequence (Predict Branch Taken), 4) Delayed Branch (define branch to take place AFTER a following instruction).
Module 4: Instruction Level Parallelism β Dynamic
Simple pipelines facilitate in-order execution, but performance enhancement requires out-of-order execution when data operands are available. Out-of-order execution may introduce data hazards of type WAR (Write After Read) and WAW (Write After Write). Instruction Level Parallelism (ILP) can be achieved by Hardware or Software.
Dynamic Scheduling techniques exploit ILP when dependencies cannot be determined at run time. It divides the ID stage into two parts: Issue the instruction in-order, and Read operand out-of-order. The two major techniques studied were Scoreboarding and Tomasulo's Algorithm.
Tomasulo's Approach for IBM 360/91 achieves high performance without special compilers. Control and buffers are distributed with Function Units (FU). Registers in instructions are replaced by values or pointers to Reservation Stations (RS) β this is register renaming. Unlike Scoreboard, Tomasulo can have multiple loads outstanding.
Hardware-based Speculation allows speculation that a branch is correctly predicted, executing out-of-order but committing in-order after confirming correctness and no exceptions exist.
| Technique | Stalls Reduced |
|---|---|
| Forwarding and bypass | Potential Data Hazard Stalls |
| Delayed Branching and Branch Scheduling | Control Hazard Stalls |
| Basic Dynamic Scheduling (Scoreboarding) | Data Hazard Stalls from true dependences |
| Dynamic Scheduling with renaming (Tomasulo) | Stalls from data hazards, anti-dependences, output dependences |
| Dynamic Branch Prediction | Control Hazard stalls |
| Speculation | Data and Control Hazard stalls |
| Multiple Instructions issue per cycle | Ideal CPI > 1 |
Module 5: Instruction Level Parallelism β Static
Multiple-instruction-issue per cycle processors are high-performance processors existing in three flavors: Superscalar Processors (exploit ILP using static and dynamic scheduling), VLIW processors (exploit ILP using static scheduling only), and Vector Processors.
| Software Scheduling Technique | Stalls Reduced |
|---|---|
| Basic Compiler scheduling | Data hazard stalls |
| Loop Unrolling | Control hazard stalls |
| Compiler dependence | Ideal CPI, Data hazard stalls |
| Trace Scheduling | Ideal CPI, Data hazard stalls |
| Compiler Speculation | Ideal CPI, Data and control hazard stalls |
Module 6: Memory Hierarchy System
The gap between processor speed and storage devices (DRAM, SRAM, Disk) increases with time. To obtain high-speed storage at cheapest cost per byte, different memory types are organized in a hierarchy based on Caching and the Principle of Locality.
π Definition β Principle of Locality: The processor accesses a relatively small portion of address space of the fastest memory closest to the processor at any instant. Temporal locality is locality in time; Spatial locality is locality in space.
π Definition β Caching: Using a small, fastest, and most expensive storage as a staging area to store frequently-used subsets of data/instructions from cheaper, larger, slower memory, avoiding main memory access every time information is needed.
Cache performance is improved using four options to reduce: miss penalty, miss penalty or miss rate via parallelism, miss rate, and time to hit in the cache.
Module 7: Multiprocessing
Parallel Architecture is a collection of processing elements that cooperate and communicate to solve larger problems faster. Four categories: SISD, SIMD, MISD, and MIMD architecture. Based on memory organization and interconnect strategy, MIMD machines are classified as Centralized Shared Memory Architecture and Distributed Memory Architecture.
Cache Coherence Problem: occurs in symmetric shared-memory multiprocessing. Two resolution methods are Write Invalidation and Write Broadcasting schemes. The Snooping Algorithm implements cache coherence using a finite state machine.
Module 8: I/O Systems
Overall computer performance is measured by throughput, heavily influenced by external systems. Neglecting I/Os is visualized by Amdahl's Law: system speedup limited by the slowest part.
I/O Interconnect Trends: Networks offer message-based narrow-pathway for distributed processors over long distance. Backplanes offer memory-mapped wide pathway for centralized processing over short distance. Channels use I/O buses; backplanes use CPU-Memory buses.
Bus Transition Protocols specify the sequence of events and timing requirements for information transfer as synchronous or asynchronous communication. Bus Arbitration Protocols reserve the bus, balancing Bus-priority (highest priority device serviced first) and Fairness (every device guaranteed eventual bus access).
Three bus arbitration schemes: Daisy Chain Arbitration, Centralized Parallel Arbitration, Distributed Arbitration.
Reliability improvement methods: Fault Avoidance (prevent fault by construction), Fault Tolerance (provide service complying with specification via redundancy), Error Removal (minimize errors by verification), Error Forecasting (estimate presence, creation, and consequence of errors by evaluation).
Module 9: Networks and Clusters
A generic interconnection network comprises computer nodes (host/end system), H/W and S/W interface, links to the interconnection network, and communication subnet. Interconnections are designated as Local Area Network (LAN), Wide Area Network (WAN), and System/Storage Area Network (SAN).
The interconnect communication model shows two machines connected via two unidirectional wires with a FIFO queue at the end. Communication software separates header and trailer, identifying request, reply, acknowledgments, and error checking codes. Communication protocols suggest the sequence of steps for reliable communication.
Distributed switch interconnects are classified as fully/partially connected and symmetric/asymmetric. Topologies include linear array, ring, 2D mesh/torus, and hypercube with their performance measures.
Internetworking connects two or more interconnection networks for reliable and efficient communication, relying on communication standards composed of hierarchy of layers. Internet communication protocol families facilitate applications to work with any interconnection.
β Key Takeaways
The most critical concepts for examination include Amdahl's Law as the fundamental performance metric where speedup equals execution time without enhancement divided by execution time with enhancement. The three pillars of computer architectureβhardware, instruction set, and softwareβwith ISA taxonomy distinguishing stack, accumulator, and general purpose register architectures. Pipeline hazards must be mastered: structural (same resource), data (invalid reads resolved by forwarding and stalling), and control (branches resolved by stalling, prediction, or delayed branching). Dynamic scheduling through Tomasulo's approach with register renaming and reservation stations enables out-of-order execution with in-order commitment through speculation. Memory hierarchy relies on the principle of locality (temporal and spatial) and caching, with four optimization options targeting miss penalty, miss rate, and hit time. For multiprocessing, the cache coherence problem requires write invalidation or broadcasting protocols implemented via snooping algorithms.
π§ Quick Revision Questions
-
What is Amdahl's Law and how does it relate to the concept that system speedup is limited by the slowest part?
-
What are the three types of pipeline hazards and what techniques are used to resolve each type?
-
How does Tomasulo's algorithm achieve out-of-order execution through register renaming and reservation stations, and what advantage does it have over Scoreboarding?
-
What is the Principle of Locality, what are its two types, and how does caching exploit this principle in memory hierarchy design?
-
What are the three bus arbitration schemes, and how do they balance bus-priority with fairness in I/O systems?