π Lecture 9 β Computer Hardware Design (Multi Cycle and Pipeline - Datapath and Control Design)
π Overview: This lecture explores the transition from single-cycle to multi-cycle datapath design, emphasizing how sharing hardware components reduces overall cost while potentially improving performance. It introduces control design methodologies using Finite State Machines and Microprogramming, and provides a foundation for understanding pipelining.
ποΈ Topics Covered
The lecture begins by recapping the single-cycle datapath and its inefficiencies, then introduces the multi-cycle approach which breaks instruction execution into five steps using shared memory and ALU. It details the multi-cycle datapath architecture, including control signals for each cycle, and compares FSM-based versus microprogrammed control design. The lecture concludes with an introduction to pipelining as a performance enhancement technique.
π Lecture Summary
Recap: Lecture 8
The single-cycle datapath executes instructions using a fixed-length clock cycle that must accommodate the longest instructionβthe Load instruction. Analysis shows that hardware components are used inefficiently: memory is used twice (instruction fetch and data access), adders in the fetch unit are used only briefly, and the ALU operates only during execution or address calculation. This underutilization leads to the conclusion that hardware can be reduced without sacrificing performance by adding more control logic.
Multiple Cycle Approach
In the multi-cycle approach, each instruction is executed in up to five sequential steps: Instruction Fetch, Instruction Decode and Register Read, Execute (for R-type, I-type, or address calculation), Memory (read/write), and Write (to register file).
π Definition β Cycle Time: The length of one clock period in a processor. In a single-cycle implementation, the cycle time equals the time to execute the longest instruction (Load). In a multi-cycle implementation, the cycle time equals the time to complete the longest single step (typically memory read/write).
π Formula: For single cycle, T_single = longest instruction time. For multi cycle, T_multi = longest step time. Example: If T_multi = 1 ΞΌSec, then T_single β 5 ΞΌSec.
π Example: Consider three instructions: Load, Store, R-type (Add). Single cycle: 3 Γ 5 ΞΌSec = 15 ΞΌSec. Multi cycle: Load takes 5 cycles, Store and R-type take 4 cycles each β 5+4+4=13 cycles Γ 1 ΞΌSec = 13 ΞΌSec. Multi-cycle is 15/13 β 1.24 times faster.
Single Cycle vs. Multiple Cycle
Multi-cycle implementations can take 3, 4, or 5 cycles per instruction depending on the instruction class. The key advantage is that cycle time is shorter because it is set to the longest step, not the longest instruction. However, more cycles are needed per instruction. For the example program segment, the multi-cycle design's shorter cycle time offsets the extra cycles, making it 1.24 times faster overall.
High Level View of Multiple Cycle Datapath
The multi-cycle datapath uses a single shared memory (since instruction fetch and data access occur in different cycles) and a single shared ALU (used for fetch, execute, and address calculation in different cycles). To support sharing, additional multiplexers and temporary registers are added: Instruction Register (IR), Memory Data Register (MDR), A and B operand registers, and ALUOut register. These hold intermediate results for use in later cycles. π‘ Why this matters: Sharing reduces hardware cost but requires careful sequencing and extra multiplexing logic.
Multiple Cycle Datapath Architecture
- Cycle 1 β Instruction Fetch: MUX-1 selects input IorD=0, connecting PC to memory read address. Instruction is fetched and stored in IR by asserting IRWr. Simultaneously, ALUSelA=0, ALUSelB=00 to add 4 to PC; PCSrc=0 and PCWr is asserted, loading PC+4.
- Cycle 2 β ID and Register Read: Instruction is decoded. Rs, Rt, Rd, and Imm16 fields become available. Registers at Rs and Rt are read onto buses A and B.
- Cycle 3 β Execute: ALUSelA and ALUSelB are set based on instruction type. For R-type: ALUSelA=1, ALUSelB=01 (bus A and B to ALU). For I-type/Memory: ALUSelA=1, ALUSelB=11 (bus A and sign-extended Imm16). For J-type: if condition test (sub) gives Zero=1, assert PCWrCond; then compute branch target: ALUSelA=0, ALUSelB=10, assert BrWr, PCSrc=1.
- Cycle 4 β Memory (Load/Store): For Load: IorD=1, passing ALUOut to memory read address; data appears at Dout. For Store: MemWr is asserted; ALUOut provides write address, bus B provides data.
- Cycle 5 β Write Back: For R-type: RegDest=1 (select Rd), MemToReg=0 (ALUOut to Bus-W), RegWr asserted. For I-type: RegDest=0 (select Rt), MemToReg=0. For Load: RegDest=0, MemToReg=1 (Dout to Bus-W), RegWr asserted.
π Definition β Control Signals: Boolean values that determine the operation of datapath components in each cycle. Examples: ALUSelA, ALUSelB, RegDest, MemToReg, RegWr, MemWr, IorD, PCWr, IRWr, PCSrc.
Multi Cycle Control Design
Control can be designed using two primary approaches:
- Finite State Machine (FSM): The sequence of control states is defined by next state functions. Logic is represented through logic equations and typically implemented with Programmable Logic Arrays (PLAs).
- Micro-programming: A micro-program counter and a dispatch ROM define the sequence of control steps. Logic is represented by a truth table and control is implemented using ROM.
Microprogram Controller
A "macroinstruction" (e.g., an assembly instruction like ADD or LOAD) is interpreted by a sequence of microinstructions stored in control memory. The microprogram controller fetches each microinstruction, executes the specified datapath control signals, and determines the next microinstruction address.
Designing a Microinstruction Set
The design process involves:
- Start with the list of all control signals.
- Group related signals into logical fields (e.g., ALU operation, ALU operands).
- Place fields in a logical order (ALU first, sequencing last).
- Create a symbolic legend for the microinstruction format, showing field names, possible values, and how they set control signals. Use computer-aided design tools.
- To minimize microinstruction width, encode operations that never occur simultaneously, using fewer bits.
π Definition β Microprogramming: A control design technique where a simple processor interprets macroinstructions by executing a sequence of microinstructions stored in control memory. It is fundamental for complex instruction sets and when few register transfers are possible per instruction.
Microprogramming: Inspiration for RISC
The lecture explains that if simple instructions can execute at very high clock rates, compilers can produce microinstructions, most programs use simple instructions, and microcode is stored in RAM for bug fixes, then why not skip instruction interpretation entirely? This reasoning leads to RISC (Reduced Instruction Set Computer) architectures, where instructions are compiled directly into the machine's lowest language, avoiding the overhead of microprogram interpretation. π‘ Why this matters: RISC philosophy emerged from recognizing that microprogramming is overkill when the ISA matches the datapath directly (1-to-1).
β Key Takeaways
The multi-cycle datapath shares memory and ALU across instruction cycles, reducing hardware cost compared to single-cycle designs while often improving performance due to shorter cycle times. Control is critical, implemented either through Finite State Machines (hardware-based logic) or microprogramming (ROM-based sequencer). The five-cycle execution model (Fetch, Decode/RegRead, Execute, Memory, WriteBack) is the foundation for understanding pipelining. Microprogramming, while powerful for complex ISAs, inspired the RISC movement by showing that simpler instructions can be executed more efficiently without micro-interpretation overhead.
π§ Quick Revision Questions
- What are the five steps in which a multi-cycle processor executes an instruction?
- Why is the multi-cycle approach faster than the single-cycle approach for the given example (Load, Store, R-type)?
- Name the three temporary registers added in the multi-cycle datapath and state the purpose of each.
- What are the two main methods for designing a multi-cycle control unit, and how do they differ in implementation?
- How did the concept of microprogramming inspire the development of RISC architectures?
π Lecture 10 β Computer Hardware Design (Pipeline Datapath and Control Design)
π Overview: This lecture introduces the fundamental concept of pipelining in computer architecture, comparing it to sequential processing using a laundry analogy. It explains the five-stage pipeline datapath, control design, performance benefits over single-cycle and multi-cycle machines, and introduces the concept of pipeline hazards that can degrade performance.
ποΈ Topics Covered
The lecture covers the concept of pipelining and its natural analogy, features of pipelined processors, the five steps of the datapath (IF, ID, EX, MEM, WB), pipelined processor design with registers and control, performance comparison between single-cycle, multi-cycle, and pipelined machines through numerical examples, pipeline execution representation, and an introduction to structural, data, and control hazards along with their impact on CPI and speedup.
π Lecture Summary
What is pipelining?
Pipelining is a fundamental concept that utilizes the capabilities of the Datapath by allowing multiple instructions to overlap in execution. It doesn't help the latency of a single task, but it improves the throughput of the entire workload. The potential speedup equals the number of pipe stages.
π Definition β Pipelining: A technique where multiple instructions are overlapped in execution by breaking them into sequential stages, each handled by independent hardware units.
Pipelining is Natural! β Laundry Example
The lecture uses a laundry example with four loads (A, B, C, D) and four operations: Wash (30 min), Dry (30 min), Fold (30 min), and Stash (30 min). In sequential laundry, each load goes through all four operations before the next load starts, taking 8 hours for 4 loads. In pipelined laundry, work starts as soon as possible β while load A is drying, load B is being washed, etc. This takes only 3.5 hours for 4 loads.
π Example: Sequential laundry for 4 loads takes 30 min Γ 4 operations Γ 4 loads = 480 min (8 hours). Pipelined laundry takes (4 loads Γ 30 min) + (3 remaining stages Γ 30 min) = 120 + 90 = 210 min (3.5 hours).
Features of Pipelined Processor
- All the functional units operate independently
- Multiple tasks operating simultaneously using different resources
- Pipelining doesn't help latency of single task, it helps throughput of entire workload
- Potential speedup = Number of pipe stages
Pipelining Lessons
- Pipeline rate is limited by the slowest pipeline stage
- Time to "fill" pipeline and time to "drain" it reduces speedup
- Unbalanced lengths of pipe stages reduces speedup
- If washer takes longer than dryer, then dryer has to wait
- Stall for Dependences is required when instructions depend on each other
Five Steps of Datapath
- Instruction Fetch (IF)
- Instruction Decode/Register Read (ID/Reg)
- Execute (EX)
- Memory Access (Mem)
- Write Back (Wr)
Pipelined Registers Included
Pipeline registers are inserted between each stage to hold intermediate results and control signals as instructions move through the pipeline stages.
Multiple Cycle vs Pipeline β Pipeline enhances performance
The lecture compares a 3-instruction program (Load, Store, R-type ADD) across three machine types.
π Example 1: The cycle time of a single cycle machine is 45 ns, and of multi-cycle and pipelined machines is 10 ns; average CPI due to instruction mix on multi-cycle machine is 4.6. What is the execution time on each type of machine for 100 instructions?
Solution:
- Single Cycle Machine: 45 ns/cycle Γ 1 CPI Γ 100 inst = 4500 ns
- Multi Cycle Machine: 10 ns/cycle Γ 4.6 CPI Γ 100 inst = 4600 ns
- Pipelined Machine: 10 ns/cycle Γ (1 CPI Γ 100 inst + 4 cycle drain) = 1040 ns
π Example 2: Consider a multicycle, unpipelined processor requires 4 cycles for the ALU and Branch operations and 5 cycles for the memory operation. Assume the relative frequency of these operations is 40%, 25% and 35% respectively; and the clock cycle is of 1 n sec. In pipelined implementation, due to clock skew and setup processor adds 0.2 n sec to the clock.
Solution:
- Unpipelined Processor: Average Execution Time/Instruction = Clock Cycle Γ Average CPI = 1 n sec Γ [{(0.4 + 0.25)} Γ 4 + 0.35 Γ 5] = 1 n sec Γ (0.65 Γ 4 + 0.35 Γ 5) = 1 n sec Γ (2.60 + 1.75) = 4.35 n sec
- Pipelined Processor: Average Execution Time/Instruction = Clock cycle + overhead = 1 n sec + 0.2 n sec = 1.2 n sec
- Speed up = 4.35 / 1.2 = 3.62 times
π‘ Why this matters: Pipelining achieves significant speedup even with a small clock overhead, because it processes one instruction per cycle after the pipeline is filled, rather than requiring multiple cycles per instruction.
Why Pipeline? Because the resources are there!
Pipelining exploits the fact that different functional units (instruction memory, register file, ALU, data memory) are idle during different phases of a single instruction's execution. Pipelining keeps these resources busy simultaneously.
Can pipelining get us into trouble?
Yes, through hazards:
- Structural hazards: Hardware cannot support the combination of instructions in the pipeline
- Data hazards: An instruction depends on the result of a previous instruction still in the pipeline
- Control hazards: Pipelining makes wrong decisions on branch predictions
π Definition β Hazard: A situation in pipelining that prevents the next instruction in the instruction stream from executing during its designated clock cycle.
How Stall degrades the performance?
- The pipelined CPI with stalls = Ideal CPI + Stall clock cycles per instruction
How Stall degrades the performance? (Speedup formulas)
- Speedup w.r.t unpipelined = CPI Unpipelined / (1 + stall cycles per instruction)
- Speedup w.r.t. pipeline depth = Pipeline depth / (1 + stall cycles per instruction)
π Formula: Speedup = Pipeline Depth / (1 + Stall Cycles per Instruction) β This shows that stalls reduce the ideal speedup equal to the number of pipeline stages.
β Key Takeaways
Pipelining dramatically improves throughput by overlapping instruction execution across multiple stages, achieving a potential speedup equal to the number of pipeline stages. The five-stage pipeline (IF, ID, EX, MEM, WB) is the standard model, and numerical examples show pipelined processors can be 3.6 to 4.3 times faster than non-pipelined alternatives. However, hazards (structural, data, and control) introduce stalls that degrade performance, and the actual speedup is reduced to (Pipeline Depth) / (1 + Stall Cycles per Instruction). The pipeline rate is always limited by the slowest stage, making balanced stage design critical.
π§ Quick Revision Questions
- What are the five stages of a pipelined datapath, and what does each stage do?
- Using the laundry analogy, explain why pipelining improves throughput but not latency of a single task.
- What is the execution time for 100 instructions on a pipelined machine with 10 ns cycle time and 4 cycle drain?
- What are the three types of pipeline hazards, and what causes each one?
- How do you calculate the speedup of a pipelined processor compared to an unpipelined processor when stalls are present?
π Lecture 11 β Computer Hardware Design (Pipeline and Instruction Level Parallelism)
π Overview: This lecture covers the three primary types of hazards that can occur in a pipelined datapath: structural, data, and control hazards. It explains the causes of these hazards and presents multiple solutions for each, including stalling, forwarding, and branch prediction techniques, which are critical for maintaining performance in modern processors.
ποΈ Topics Covered
This lecture begins with a recap of multi-cycle versus pipeline datapaths and performance enhancement. It then delves into structural hazards, caused by resource conflicts like a single memory port, and presents solutions like inserting bubbles or adding functional units. Next, it covers data hazards, which arise from instruction dependencies, and explains solutions like stalling and forwarding/bypassing, with special attention to loads. Finally, it addresses control hazards caused by branch instructions, detailing several handling methods including stalling, re-fetching, delayed branching, and branch prediction.
π Lecture Summary
Recap: Lecture 10
The lecture reviews the transition from a multi-cycle datapath to a pipelined datapath and the key components involved. It reiterates how pipelining enhances performance and introduces the concept of hazards that can disrupt the pipeline flow.
Structural Hazards
A structural hazard occurs when the hardware attempts to use the same resource in two different ways at the same time. For example, a single memory port being accessed for both an instruction fetch and a data read in the same clock cycle creates a structural hazard.
π‘ Why this matters: Structural hazards force the pipeline to stall, reducing the performance benefit of pipelining. A common example is a LOAD instruction accessing data memory while the next instruction is being fetched from the same single-ported memory. This conflict is resolved by inserting a stall (or a "bubble") into the pipeline.
π Definition β Structural Hazard: An attempt to use the same resource two different ways at the same time. π Solution: Insert a "bubble" (stall) or add multiple functional units (e.g., dual-ported memory).
Pipelining the Load Instruction The five independent functional units used per instruction are: Instruction Memory (IF), Register File's Read ports (ID), ALU (EX), Data Memory (MEM), and Register File's Write port (WB). Because each unit is used only once per instruction, no conflicts arise when pipelining only load instructions. Performance improves dramatically; three loads take 7 cycles pipelined versus 15 cycles in a multi-cycle processor, achieving an effective CPI of 1.
The Four Stages of R-type An R-type instruction does not access data memory, so it completes in four stages: IF, ID, EX, and WB. During the EX stage, the ALU operates on register operands, and the result is written back in the WB stage.
π R-type instruction: An instruction that uses registers for all operands and results, completing in four stages (IF, ID, EX, WB) in a 5-stage pipeline.
Pipelining the R-type and Load Instruction Mixing R-type and Load instructions creates a structural hazard because the R-type instruction writes to the register file in its 4th stage (WB), while the Load instruction writes in its 5th stage (WB). This leads to two instructions trying to write to the single write port of the register file at the same time.
π Important Observation: Each functional unit must be used at the same stage for all instructions to prevent structural hazards.
Solution 1: Insert "Bubble" into the Pipeline Insert a stall cycle after the Load instruction to delay the R-type instruction's write, preventing the conflict. This solution is complex to implement and degrades performance because the effective CPI for a load becomes 2.
Solution 2: Delay R-type's Write by One Cycle Add a "No-OP" (MEM) stage to the R-type instruction pipeline, making it take 5 stages (same as Load). The R-type now uses the register file's write port at its 5th stage, eliminating the conflict. This simpler solution maintains an effective CPI of 1.
Eliminating Structural Hazards Structural hazards can be minimized by using stalls (bubbles) or by adding multiple functional units (e.g., separate instruction and data caches, or dual-ported register files).
π Example: Dual-port vs. Single-port
- Machine A: Dual-ported memory, ideal CPI = 1.
- Machine B: Single-ported memory, 1.05x faster clock, ideal CPI = 1, loads are 40% of instructions.
- SpeedUp_A = Pipeline Depth
- SpeedUp_B = (Pipeline Depth / (1 + 0.4)) * 1.05 = 0.75 * Pipeline Depth
- SpeedUp_A / SpeedUp_B = 1.33, meaning Machine A is 1.33 times faster.
Data Hazards
A data hazard occurs when an instruction attempts to use an operand before the previous instruction that produces it is ready. This happens because pipelining overlaps instruction execution, changing the relative timing of read/write operations.
π Definition β Data Hazard: A situation where an instruction depends on the result of a prior instruction still in the pipeline, occurring because of a change in the order of operand read/write.
Example Data Hazard on R1
Add R1, R2, R3 ; R1 = R2 + R3
Sub R4, R1, R3 ; Needs R1 from Add
And R6, R1, R7 ; Needs R1 from Add
Or R8, R1, R9 ; Needs R1 from Add
Xor R10, R1, R11 ; Needs R1 from Add (safe if after write)
Data Hazard Solution #1 β Stall Insert stalls into the pipeline to wait until the required data is available. This effectively "freezes" the dependent instructions until the producing instruction's result is written back.
Data Hazard Solution β Forwarding (or Bypassing) Also known as forwarding, this technique resolves data hazards by directly feeding the result from a previous instruction's EX or MEM stage to the dependent instruction's ALU input, bypassing the register file. This avoids stalls for ALU-to-ALU dependencies.
Forwarding (or Bypassing): What about Loads? Load instructions introduce a "use" data hazard because the data from memory (MEM stage) is needed in the next instruction's EX stage. Since the data is not available early enough for forwarding, a stall (often called a "load-use stall") is required when an instruction immediately uses the result of a load.
Control Hazards
A control hazard (or branch hazard) occurs when the pipeline makes a decision about which instruction to fetch next before the condition of a branch instruction is evaluated.
π Definition β Control Hazard: A hazard that arises when a branch instruction may change the Program Counter (PC) to something other than PC+4, and the next instruction has already been fetched.
Example BEQ Taken If a branch instruction (e.g., BEQ) is taken, the next instruction address is determined in the EX stage. However, the next instruction (a LOAD) is already fetched in the ID stage, leading to a control hazard. The simple solution is to freeze the pipeline (hold instructions after the branch) and flush (delete) incorrectly fetched instructions if the branch is taken.
Dealing with Branches There are five main methods:
- Stall
- Redo Fetch after branch
- Delayed branching
- Branch prediction
- Multiple Streams
Solution #1: Stall The simplest approach. Stall the pipeline until the branch condition is evaluated (EX stage) and the target address is known (next stage). This introduces 3 stall cycles per branch.
Reducing Number of Stall By moving the branch decision logic to the end of the ID stage (2nd stage) and adding hardware to compare registers, the number of stalls can be reduced to 2 clock cycles per branch instruction. This can be further reduced to 1 for BEQZ/BNEZ if a zero register is tested.
Solution #2: Redo Fetch after Branch Since a branch is detected in the ID stage, the next instruction fetch is a stall. The incorrectly fetched instruction is ignored, and the correct successor instruction is refetched. The impact is 1 clock cycle per branch instruction if the branch is untaken.
Solution #3: Delayed Branch β S/W Method This software technique redefines the branch behavior to take place after the next instruction (the delay slot). The instruction in the delay slot is always executed, regardless of whether the branch is taken or not. The impact is 0 clock cycles per branch if a useful instruction can be placed in the slot (approximately 50% of the time).
Solution #4: Prediction The processor guesses the direction of the branch and starts fetching from that path. If the guess is wrong, the pipeline is flushed and restarted.
- Predict Branch Not Taken: Assume the branch is not taken and continue fetching sequentially. If taken, flush the fetched instructions and fetch from the target.
- Predict Branch Taken: Assume the branch is taken and start fetching from the target address as soon as it's computed. This is more effective for loops where the branch is taken most of the time.
Solution #5: Multiple Streams Have two separate pipelines. One pipeline pre-fetches instructions sequentially, while the other pre-fetches from the branch target. The appropriate pipeline is used when the branch is resolved. This leads to bus and register contention and is more complex.
β Key Takeaways
The three primary hazards in pipelined datapaths are structural (resource conflicts), data (operand dependencies), and control (branch decisions). Structural hazards are solved by stalling or adding more hardware. Data hazards are mitigated using forwarding for ALU results, but a stall is still needed for "load-use" dependencies. Control hazards can be handled through stalls, delayed branching, or sophisticated branch prediction schemes, which are essential for high-performance processors. The goal of all these techniques is to maintain an effective CPI as close to 1 as possible by minimizing pipeline stalls.
π§ Quick Revision Questions
- What is the primary difference between a structural hazard caused by a single memory vs. a data hazard?
- Explain the concept of "forwarding" as a solution for data hazards and why it is not always sufficient for load instructions.
- What are the performance implications (in terms of stall cycles) for the "stall" and "delayed branch" solutions to control hazards?
- Describe the "predict branch not taken" strategy for handling control hazards. What happens if the prediction is wrong?
- Why is it necessary for every instruction in a pipeline to use each functional unit at the same stage to avoid structural hazards?
π Lecture 12 β Instruction Level Parallelism (Introduction to multi cycle pipelined datapath)
π Overview: This lecture extends basic pipelining concepts to handle floating-point (FP) instructions with longer latencies in MIPS architecture. It explores how instruction-level parallelism (ILP) can be enhanced through deeper pipelines, examines the hazards that arise from longer FP operations, and introduces both static and dynamic scheduling techniques to overcome these hazards.
ποΈ Topics Covered
The lecture begins with a recap of pipelining basics and three generic data hazards (RAW, WAR, WAW), then moves into longer pipelines specifically for FP instructions. It covers loop-level parallelism, FP loop hazards including structural, data, and control hazards, and presents the MIPS FP adder flow diagram with step-by-step addition. The lecture details functional unit latency and initiation intervals, shows two versions of MIPS FP pipelines (with unpipelined and pipelined functional units), and concludes with scheduling techniques and exception handling in out-of-order execution.
π Lecture Summary
Recap: Pipelined datapath and control
The previous lecture reviewed the pipelined datapath to understand ILP basics β overlapping instruction execution to enhance performance. Key components of the pipeline datapath were examined. Performance enhancement from pipelining helps instruction bandwidth but not latency. Three types of pipeline hazards were identified: structural hazards, data hazards, and control hazards.
π Definition β Pipeline hazard: A situation that prevents the next instruction in the instruction stream from executing during its designated clock cycle.
Three Generic Data Hazards
Read After Write (RAW): A true data dependence where instruction J tries to read an operand before instruction I writes it. Example: i: add r1,r2,r3 followed by j: sub r4,r1,r3 (j needs r1 before i writes it).
Write After Read (WAR): An anti-dependence (name dependence) where instruction J writes to a register before instruction I reads it. Example: i: sub r4,r1,r3 followed by j: add r1,r2,r3. This can be resolved through register renaming.
Write After Write (WAW): A name dependence where instruction J writes to a register before instruction I writes to it. Example: i: sub r1,r4,r3 followed by j: add r1,r2,r3.
π Definition β Name dependence: A hazard that occurs when two instructions use the same register or memory location but do not have a true data flow relationship; can be eliminated by register renaming.
How to remove Hazards?
- Structural Hazard: Use multiple functional units to avoid resource conflicts.
- Data Hazard: Use forwarding or bypassing to send results directly to dependent instructions without waiting for register write.
- Control Hazards: Use predict (branch prediction) or delay branch (branch delay slots) techniques.
Instruction Level Parallelism
ILP is achieved through two main factors: clock speed and the number of instructions that can execute in parallel. A superscalar processor achieves ILP by pre-fetching and decoding multiple instructions, starting several branch instruction streams, and finally discarding all but the correct stream.
π Definition β Instruction Level Parallelism (ILP): The degree to which instructions in a program can be executed simultaneously or overlapped.
MIPS Longer Pipelines β FP Instructions
For FP addition, minimum four steps are performed. The MIPS FP Adder flow diagram shows the processing stages.
Steps for FP Addition:
- Step 1: Compare exponents of two numbers; shift the smaller number to the right until its exponent matches the larger exponent.
- Step 2: Add the significands.
- Step 3: Normalize the sum β shift right and increment or shift left and decrement.
- Step 4: If no overflow or underflow, round the significand to the required number of bits. Stop if further normalization is not required; otherwise go to step 3.
Latency of Functional Unit
π Definition β Latency: The number of cycles between the instruction that produces a result and the one that uses the result of the operation.
π Definition β Initiation (Repeat) Interval: The number of cycles that must elapse between issuing two operations of the same type.
π Latency and Initiation Interval Table:
| Functional Unit | Latency | Initiation Interval |
|---|---|---|
| Integer ALU | 0 | 1 |
| Data Memory (Int/FP Load) | 1 | 1 |
| FP ADD | 3 | 1 |
| FP/Integer Multiply | 6 | 1 |
| FP/Integer Divide | 24 | 25 |
π‘ Why this matters: The FP divide has an initiation interval of 25 cycles, meaning you cannot issue another divide operation for 25 cycles after starting one, creating a structural hazard.
Typical MIPS FP Pipeline
A typical MIPS FP pipeline uses three un-pipelined FP functional units. This means each functional unit (FP adder, FP multiplier, FP divider) processes one instruction at a time without internal pipelining.
MIPS FP Pipeline with Pipelined FUs
The previous FP pipeline can be extended by adding additional pipeline stages in the functional units. Additional pipeline registers are inserted between intervening stages (e.g., A1/A2, A2/A3, etc.). The ID/EX register must be expanded to connect the ID stage to A1, M1, EX, and DIV functional units. The FP divide unit is not pipelined and requires 24 clock cycles to complete.
Hazards in Longer Latency Pipeline
- Structural hazard: All functional units are not fully pipelined, so resource conflicts may occur.
- WAW data hazard: Instructions have varying running time and are no longer reaching the WB stage in order, so WAW hazards may occur.
- WAR hazards are not possible because registers are read in the ID stage before any writes occur.
- RAW data hazard stalls may be more frequent because of longer latency of operations.
π Example β FP Pipeline RAW Hazard:
Clock: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
L.D F4,0(R2): IF ID EX Me WB
MUL.D F0,F4,F6: IF ID st M1 M2 M3 M4 M5 M6 M7 Me WB
ADD.D F2,F0,F8: IF st ID st st st st st st A1 A2 A3 A4 Me WB
S.D F2,0(R2): IF st st st st st st ID st st st EX Me
The MUL.D must stall for L.D result (RAW hazard), and ADD.D stalls for MUL.D result, causing multiple stall cycles.
π Example β FP Pipeline Structural Hazard:
Clock: 1 2 3 4 5 6 7 8 9 10 11
MUL.D F0,F4,F6: IF ID M1 M2 M3 M4 M5 M6 M7 Me WB
... IF ID Ex Me WB
... IF ID Ex Me WB
ADD.D F2,F0,F8: IF ID A1 A2 A3 A4 Me WB
... IF ID Ex Me WB
... IF ID Ex Me WB
L.D F4,0(R2): IF ID EX Me WB
Multiple instructions compete for the same functional unit, causing structural hazards.
Conclusion about FP Pipeline
- Structural Hazard: Wait until the required functional unit is available.
- RAW Data Hazard: Wait until source registers are not listed as pending destination registers that will not be available.
- WAW Hazard: Determine if any instruction in A1, A2, ... D, M1, M2, ... has the same destination as this instruction.
Precise Exceptions: Out-of-order Completion
In the program:
DIV.D F0,F2,F4
ADD.D F10,F10,F8
SUB.D F12,F12,F14
Instructions complete out of order, causing potential exception problems.
π Definition β Precise exception: An exception where all instructions before the faulting instruction have completed and none after have modified any state, allowing the program to restart cleanly.
Overcoming the Data Hazard by Scheduling
- Static Scheduling: Compiler-based scheduling where the compiler rearranges instructions to avoid hazards.
- Dynamic Scheduling: Hardware-based scheduling that reorders instructions at runtime.
Dynamic Scheduling Advantages
- Allows handling cases where dependence is unknown at compile time.
- Allows code compiled for one pipeline to run on another pipeline.
In the program:
DIV.D F0,F2,F4
ADD.D F10,F0,F8
SUB.D F12,F8,F14
Problems of Out-of-order execution: WAR and WAW
In the program:
DIV.D F0,F2,F4
ADD.D F6,F0,F8
SUB.D F8,F10,F14
MUL.D F6,F10,F8
Out-of-order execution introduces WAR and WAW hazards that must be managed.
Overcoming Exceptions
Split the ID pipe stage into two:
- Issue: Decode instructions and check for structural hazard.
- Read Operand: Wait until no data hazards exist, then read operands.
β Key Takeaways
The most critical concept is understanding that longer FP pipelines introduce new challenges beyond basic integer pipelines: RAW stalls become more frequent due to increased latencies, WAW hazards emerge because instructions complete out of order (unlike integer pipelines where all reach WB in order), and structural hazards arise when functional units like FP divide are not fully pipelined. The latency and initiation interval values (FP ADD=3/1, FP Multiply=6/1, FP Divide=24/25) are essential for hazard analysis. To overcome these hazards, two scheduling approaches exist: static (compiler-based) and dynamic (hardware-based), with dynamic scheduling splitting the ID stage into Issue and Read Operand to handle unknown dependencies and enable cross-platform compatibility. Finally, out-of-order completion creates exception-handling challenges requiring precise exception mechanisms.
π§ Quick Revision Questions
- What are the three generic data hazards, and which one represents a true data dependence?
- What is the difference between latency and initiation interval for a functional unit?
- Why do WAW hazards occur in longer FP pipelines but not in simple integer pipelines?
- In the MIPS FP adder, what are the four steps for adding two floating-point numbers?
- What are the two advantages of dynamic scheduling over static scheduling?