π Lecture 17 β Instruction Level Parallelism (High-performance Instructions delivery - Multiple Issue)
π Overview: This lecture explores advanced techniques for delivering instructions to high-performance processors, focusing on integrated branch prediction and multiple instruction issue mechanisms. It explains how modern processors overcome the limitations of single-issue pipelines through superscalar and VLIW architectures, with detailed analysis of Tomasulo's algorithm extended for dual-issue execution.
ποΈ Topics Covered
The lecture covers branch penalty reduction through Branch Target Buffer, Integrated Instruction Fetch Units, and Return Address Predictors. It then introduces multiple instruction-issue processors, comparing superscalar and VLIW approaches. The statically scheduled superscalar MIPS processor is examined with an example pipeline, followed by dynamic scheduling in superscalar processors using extended Tomasulo's concept. Two detailed examples demonstrate dual-issue execution loops with resource usage tables and performance metrics.
π Lecture Summary
Reducing branch penalties for High-Performance Processors
Control hazards limit instruction throughput in pipelined processors. Three key hardware techniques reduce branch penalties: Branch Target Buffer stores previously taken branch addresses; Integrated Instruction Fetch Units combine branch prediction with instruction prefetch; and Return Address Predictors handle indirect jumps from procedure calls and select statements.
π Definition β Branch Target Buffer (BTB): A hardware structure that caches the target address of previously taken branches to predict the next instruction address. π Formula: No explicit formula β BTB operates as a cache lookup. π Example: When a branch instruction is fetched, the BTB is checked; if a match is found (hit), the stored target address is used as the next PC, avoiding pipeline bubbles.
Integrated Instruction Fetch Units (IIFU)
The IIFU integrates three critical functions for high-performance instruction delivery. Integrated Branch Prediction places the predictor inside the fetch unit to drive the fetch pipeline proactively. Instruction Prefetch maintains a queue holding multiple instructions, enabling delivery of more than one instruction per cycle. Instruction memory access and buffering handles the complexity of fetching across multiple cache lines and hides the cost of crossing block boundaries.
π‘ Why this matters: IIFUs are essential for superscalar processors that must deliver multiple instructions per cycle, as they prevent the fetch stage from becoming a bottleneck.
π Definition β Instruction Prefetch Queue: A buffer within the IIFU that holds fetched instructions before they are issued, enabling multiple instructions to be delivered in one clock cycle. π Formula: No formula β queue depth determines lookahead capability. π Example: In a 2-issue processor, the IIFU fetches 64 bits per clock cycle (two 32-bit MIPS instructions), buffering them for simultaneous issue to functional units.
Return Address Predictors
Return Address Predictors specialize in predicting indirect jumps whose target addresses vary at runtime. These are generated by high-level language constructs such as indirect procedure calls and switch/case statements. The predictor typically uses a hardware stack that stores return addresses when CALL instructions execute.
π Definition β Return Address Stack: A specialized hardware structure that pushes the return address on each procedure call and pops it on return, achieving near-perfect prediction for nested procedure calls.
π Example: In a recursive function call, each JAL instruction pushes the return address onto the hardware stack; subsequent JR instructions pop the correct address, avoiding pipeline stalls.
Multiple Instruction-Issue Processors
All previously described single-issue pipelines achieve at most 1 instruction per cycle (IPC). To exceed this limit, two approaches exist: Superscalar processors and Very Long Instruction Word (VLIW) processors. Superscalar processors execute multiple instructions per cycle using hardware dynamic scheduling, while VLIW processors rely on the compiler to pack independent operations into wide instruction words.
π Definition β Superscalar Processor: A processor that can issue multiple independent instructions from a conventional instruction stream in each cycle, using hardware to detect and resolve dependencies. π Formula: IPC = Instructions Executed / Cycles Taken β For a 2-issue superscalar, theoretical maximum IPC = 2. π Example: IBM Power2, Sun Ultra SPARC, Pentium III/4, DEC Alpha, and HP 8000 all implement superscalar execution.
VLIW Processors
Very Long Instruction Word (VLIW) processors bundle multiple independent operations into a single long instruction word. They incorporate advanced features like predication, rotating registers, and speculation to expose instruction-level parallelism. Typical implementations include Intel i860, Trimedia, and Itanium.
π Definition β Predication: A technique where instructions are conditionally executed based on a predicate flag, eliminating branches and reducing control hazards.
π Formula: No formula β predication uses conditional execution bits within each VLIW instruction.
π Example: A VLIW instruction r1 = L r4; r2 = Add r1,M; f1 = Mul f1,f2; r5 = Add r5,4 executes four independent operations simultaneously in one cycle.
Statically Scheduled Superscalar MIPS Processor
As an example, consider a MIPS superscalar that issues 2 instructions per cycle: one FP operation and one Integer operation (load/store to integer/FP register, branch, or integer ALU). This requires fetching and decoding 64 bits per clock cycle. Hazard detection is simplified by the restriction that only one FP and one integer instruction can issue per packet.
π Definition β Issue Packet: The group of instructions fetched simultaneously that could potentially issue in the same clock cycle, ranging from 1 to 4 instructions.
π Example: If the issue packet contains ADD.D F4,F0,F2 and S.D F4,0(R1), a RAW hazard exists because the store depends on the add. The simple solution is to issue only one instruction, treating it as a structural hazard.
The issue process follows three steps:
- Fetch two instructions from the cache
- Determine whether 0, 1, or 2 instructions can issue
- Issue them to the correct functional units
Dynamic Scheduling in Superscalar Processors
Extending Tomasulo's algorithm to support dual-issue requires careful handling. Instructions should not be issued to reservation stations out of order to avoid violating program semantics. Two approaches enable simultaneous issue: running the reservation station assignment in half a clock cycle, or building logic to handle two instructions with dependencies. Modern processors issue 4+ instructions per cycle by combining both approaches with hardware-based speculation.
Example 1: Dual-Issue Tomasulo Loop Execution
Consider the loop adding scalar F2 to array elements (from Lecture 16):
Loop: L.D F0,0(R1) ; F0=array element
ADD.D F4,F0,F2 ; add scalar in F2
S.D F4,0(R1) ; store result
DADDUI R1,R1,#-8 ; decrement pointer
BNE R1,R2,LOOP ; branch R1!=R2
Using a 2-issue Tomasulo pipeline:
- Both FP and integer operations can issue every clock cycle
- One integer ALU for ALU+address calculation; separate FP functional units
- Issue=1 cycle, Write result=1 cycle
- Latencies: Integer ALU=1 cycle, Load=2 cycles, FP add=3 cycles
π‘ Why this matters: The dual-issue pipeline achieves an IPC of 5/3 β 1.67 (5 instructions in 3 clocks per iteration), significantly better than single-issue.
π Example: The resource usage table shows:
- Clock 1:
L.DandDADDUIissue simultaneously - Clock 2:
ADD.DandBNEissue;L.Dexecutes - Clock 3:
S.Dissues;ADD.Dexecutes - Completion rate = 15/16 = 0.94 (15 instructions execute in 16 cycles)
Example 2: Overcoming Single Integer Pipe Bottleneck
With a wider CDB (2 CDBs) and separate functional units for Integer ALU and effective address calculation, the same loop executes more efficiently. This configuration eliminates the bottleneck caused by a single integer unit sharing both ALU and address computation.
π Example: The activity table for Example 2 shows improved resource utilization where integer ALU operations and effective address calculations proceed in parallel, reducing the total execution time for the loop.
β Key Takeaways
Integrated Instruction Fetch Units combine branch prediction, prefetch queues, and memory access buffering to deliver multiple instructions per cycle while hiding cache block crossing penalties. Superscalar processors achieve higher IPC through hardware dynamic scheduling, with dual-issue Tomasulo pipelines achieving 1.67 instructions per cycle for typical loops. VLIW processors shift complexity to the compiler, bundling independent operations into wide instruction words with features like predication and speculation. The key performance gains come from separating functional units (integer ALU vs. address calculation) and widening CDB bandwidth to handle multiple results per cycle. Hardware-based speculation integrates branch prediction into dynamic scheduling pipelines, enabling sustained instruction throughput beyond single-issue limits.
π§ Quick Revision Questions
- What are the three main hardware techniques for reducing branch penalties in high-performance processors?
- Explain the three integrated functions of an IIFU and how they collectively improve instruction delivery.
- How does a Return Address Stack predict indirect jumps, and what type of program constructs generate these jumps?
- Compare the key differences in instruction scheduling between superscalar and VLIW processors.
- In the dual-issue Tomasulo example, what causes the completion rate of 0.94 (15/16), and how does wider CDB improve this?
π Lecture 18 β Instruction Level Parallelism (Hardware-based speculations and exceptions)
π Overview: This lecture focuses on hardware-based speculation as a technique to overcome control hazards and improve instruction-level parallelism. It explains how extending Tomasuloβs algorithm with a Reorder Buffer (ROB) allows speculative execution, in-order commitment, and proper handling of exceptions. This is critical for modern superscalar processors to achieve higher performance by executing instructions before branch outcomes are known.
ποΈ Topics Covered
The lecture begins with a recap of branch target buffers, integrated instruction fetch units, return address predictors, and superscalar/VLIW processors. The main focus is on introducing hardware-based speculation, explaining how it combines dynamic branch prediction, speculation, and dynamic scheduling. The implementation details focus on the Reorder Buffer (ROB) and its fields, the extended Tomasuloβs algorithm with four steps (Issue, Execute, Write Result, Commit), and two detailed examples showing status tables for speculative vs. non-speculative execution. The topic of handling exceptions in speculative hardware is also discussed, including how interrupts and mispredictions are managed.
π Lecture Summary
Hardware-based Speculation: Introduction
Hardware-based speculation offers many advantages. It can incorporate hardware-based branch prediction, does not require additional bookkeeping code, and does not depend on a compiler. This approach has been implemented in processors like the PowerPC 620, MIPS R10000, Intel P6, and AMD K5.
Hardware Based Speculation: Basics
Exploiting more instruction-level parallelism increases the burden of maintaining control dependence. While branch prediction reduces direct stalls, a multiple-issue processor may need to execute a branch every clock cycle. Hence, overcoming control dependence limitations requires speculation on the outcome of branches and executing the program for speculations. This means we fetch, issue, and execute instructions as if our branch predictions were always correct. Dynamic scheduling without speculation fetches and issues instructions but does not execute them until the prediction is checked.
Hardware Support: Speculative Execution
The main idea is to allow execution of an instruction dependent on a predicted-taken branch such that there are no consequences (including exceptions like memory violation) if the branch is not actually taken. This is achieved if hardware support for speculation buffers the results and exceptions from instructions until it is known that the instruction would execute.
Hardware Based Speculation: Basics
Hardware-based speculation combines three key ideas:
- Dynamic Branch Prediction: Facilitates choosing which instruction to execute (next in sequence or branch).
- Speculation: Allows execution of instructions before the control dependence is resolved. The hardware must have the ability to undo the instructions if there are exceptions.
- Dynamic scheduling: Deals with scheduling different combinations of basic blocks.
Thus, the hardware-based speculation follows the predicted flow of data values to choose when to execute. To do this, we must separate the bypassing of results among instructions from the actual completion of an instruction. By making this separation, we can allow an instruction to execute and bypass its result to other instructions without allowing it to perform any update that cannot be undone, until we know it is no longer speculative. When the instruction is no longer speculative, we allow it to update the register file or memory. This additional step in the instruction execution sequence is called instruction commit. The basic idea is to allow instructions to execute out-of-order but force them to commit in-order.
π‘ Why this matters: This separation is the core mechanism that makes speculation safe, preventing incorrect results from speculative code from corrupting the architectural state.
Hardware Based Speculation: Implementation
In a single-issue five-stage pipeline, we can ensure in-order commitment by moving writes to the end of the pipeline. When adding speculation, we need to separate the process of completing execution and instruction-commit, as instructions may finish execution considerably before they are ready to commit. Adding this commit phase requires changes to the sequence and an additional set of hardware buffers (the Reorder Buffer) that holds the result of instructions that have finished execution but have not committed.
Modified hardware including ROB
The figure (provided in the text) shows the modified hardware. Here, the Reorder Buffer (ROB) can be an operand source if the value is not yet committed. Once an operand commits, the result is found in the register file.
Mechanism
- At issue time, allocate an entry in the ROB to hold the result.
- Use the ROB entry number instead of a reservation station to rename registers.
- However, we can use additional registers for renaming, and the ROB only for tracking commits.
- Instruction results commit to the register set in-order.
- If the ROB is implemented as a queue, it is simple to undo speculated instructions on mispredicted branches or on exceptions; it just requires throwing away uncommitted entries.
Extended Tomasuloβs Pipe
Exceptions are not recognized until an instruction becomes ready to commit. The figure shows the Tomasulo's hardware structure including the ROB for a MIPS FP unit, which is extended to handle speculation. The ROB provides additional buffer, in the same way as reservation stations in Tomasulo's, that extends the register set. The ROB holds the result of an instruction between the time the operation completes and the time the instruction commits. The ROB is a source of operands for instructions, just as reservation stations provided operands in Tomasulo's algorithm.
π Definition β Reorder Buffer (ROB): A hardware buffer that holds the results and exceptions from instructions until it is known that the instruction will commit. It allows for in-order commitment while allowing out-of-order execution.
ROB consists of four fields
- Instruction Type field: Indicates whether the instruction is a branch (no destination), a store (memory address destination), or a register operation (ALU or load, with register destination).
- Destination field: Supplies the register number (for load and ALU operations) or the memory address (for stores) where the instruction result should be written.
- Value field: Used to hold the value of the instruction result until the instruction commits.
- Ready field: Indicates that the instruction has completed execution and the value is ready.
Speculative Tomasuloβs Algorithm
There are four steps:
- Issue: Get instruction from the head of the instruction queue. If a reservation station and ROB slot are free, allocate and issue the instruction. If operands are available, send them to the reservation station; otherwise, keep track of the ROB entry that will produce the operands.
- Execute: Operate on operands. If both operands are ready, execute. If not ready, watch the CDB for the result. This checks for RAW hazards. Instructions may take multiple clock cycles here.
- Write result: Finish execution (WB). Write on the Common Data Bus (CDB), mark the reservation station available. The result is picked up by the ROB entry. For a store, if the value to be stored is available, it is written to the value field of the ROB entry; if not, the CDB is monitored until the value is broadcast.
- Commit: This can occur when an instruction reaches the head of the ROB and its result is present in the buffer. The commit updates the register or stores to memory with the ROB result and frees the ROB slot. If the ROB head is an incorrectly predicted branch, the ROB is flushed. If the branch was correctly predicted, the branch is finished.
Example 1
Using the same code segment as in the Tomasulo's approach, the status table is shown when the MUL.D is ready to go to commit. The latencies are: add=2, multiply=10, divide=40 clock cycles. The code is:
L.D F6,34(R2)L.D F2,45(R3)MUL.D F0,F2,F4SUB.D F8,F6,F2DIV.D F10,F0,F6ADD.D F6,F8,F2
The table shows that although SUB.D has completed execution, it does not commit until MUL.D commits. The reservation station and register status fields contain the same basic information as for Tomasulo's algorithm. At the time MUL.D is ready to execute, only two L.D instructions have committed. SUB.D and ADD.D will not commit until MUL.D commits. DIV.D is in execution but has not completed due to its longer latency.
Comparing with the non-speculation table, in the non-speculation case, ADD.D and SUB.D completed out-of-order (before MUL.D). In the speculative hardware: the reservation station numbers are replaced with ROB entry numbers in Qj, Qk, and in register status fields, and a destination field is added to the reservation station designating the ROB number that is the destination for the result.
π Example: The specific code and resulting status table (from the lecture text) demonstrates that in speculation, SUB.D and ADD.D are not allowed to commit (update registers) until MUL.D commits, even though their execution is complete. This preserves the original program order.
Multiple issue with speculation
A speculative processor can be extended to multiple issue. The two challenges are:
- Instruction issue and monitoring the CDBs for instruction completion.
- Maintaining throughput of greater than one instruction per cycle.
Example: Loop Execution with and without Speculation
Consider the loop:
Loop:
LD R2,0(R1) ; R2= array element
DADDUI R2,R2,#1 ; increment R2
SD R2,0(R1) ; store result
DADDUI R1,R1,#4 ; increment pointer
BNE R2,R3,LOOP ; branch if not last element
- Without speculation: The
LDfollowingBNEcannot start execution earlier because it must wait until the branch outcome is determined. - With speculation: The
LDfollowingBNEcan start execution early because it is speculative. Comparing the two tables, the third branch in the speculative processor executes in 13 clock cycles, while in the non-speculative processor it executes in 19 clock cycles. The non-speculative pipeline falls behind the issue rate rapidly.
Exceptions to Hardware-based speculation
In the Tomasulo structure with hardware-based speculation, if an exceptional situation occurs while executing an instruction, the ROB doesn't commit and handles exceptions. For example, if MUL.D causes an interrupt, we can wait until this interrupt reaches the head of the ROB, any pending instruction is flushed out, and the speculation is un-done. In the case of dynamic scheduling without speculation, the results in registers F8 (for SUB.D) and F6 (for ADD.D) could be overwritten out-of-order, and the interrupt could not be handled.
Furthermore, exceptions are handled by not recognizing them until the instruction is ready to commit. For example, in a loop, if an exception arises from MUL.D, it is recorded in the ROB. If a misprediction also arises from a speculated BNE, the exception is flushed out along with the speculated instruction when the ROB is cleared.
β Key Takeaways
The most critical idea from this lecture is that hardware-based speculation, implemented through an extended Tomasulo's algorithm with a Reorder Buffer (ROB), allows a processor to execute instructions out-of-order and speculatively (based on branch prediction) while maintaining precise exceptions and in-order commitment. This solves the control hazard problem more aggressively than simple branch prediction because the ROB buffers results until the branch is resolved, allowing execution to proceed without stalling. The key difference from non-speculative dynamic scheduling is the separation of execution from commitment, where the ROB holds all uncommitted results and is flushed on a misprediction. Finally, handling exceptions requires waiting until the faulting instruction reaches the head of the ROB to ensure all previous instructions have committed correctly, and a misprediction can flush a pending exception, which is correct because the faulty instruction should not have been executed.
π§ Quick Revision Questions
- What are the three key ideas combined in hardware-based speculation?
- What is the purpose of the Reorder Buffer (ROB), and what are its four fields?
- Describe the four steps of the speculative Tomasulo's algorithm (Issue, Execute, Write Result, Commit).
- In the Example 1 status table, why do
SUB.DandADD.Dnot commit until afterMUL.Dcommits, even though they have finished executing? - How are exceptions and branch mispredictions handled in a processor with hardware-based speculation, and what happens to the ROB?
π Lecture 19 β Instruction Level Parallelism (Limitations of ILP and Conclusion)
π Overview: This lecture concludes the discussion on dynamic scheduling techniques for Instruction Level Parallelism by introducing an ideal processor model to study the limitations of ILP. It examines how window size, branch prediction accuracy, and finite registers constrain realizable processor performance, and concludes with an analysis of the Intel P6 micro-architecture and its evolution into the NetBurst architecture of the Pentium 4.
ποΈ Topics Covered
Recap of ILP dynamic scheduling concepts including scoreboarding, Tomasulo's algorithm, branch prediction, and multiple instruction issue. Introduction of an ideal processor model to study ILP limitations. Analysis of performance limiting factors including window size and issue count, branch/jump prediction, and finite registers. Examination of performance data from the Alpha 21264 processor using SPEC92 benchmarks. Discussion of Intel P6 micro-architecture implementation in Pentium Pro, Pentium II, and Pentium III, followed by comparison with Pentium 4's NetBurst architecture.
π Lecture Summary
Recap: ILP-Dynamic Scheduling
The last few lectures covered hardware methods for instruction level parallelism to execute multiple instructions in pipelined datapaths, referred to as dynamic scheduling techniques. These techniques avoid structural, data, and control hazards while minimizing stalls. The scoreboarding and Tomasulo's algorithm were discussed as basic concepts for dynamic scheduling in integer and floating-point datapaths, facilitating out-of-order execution to minimize data dependencies. Branch-prediction techniques and different types of branch predictors were also covered to reduce stalls due to control hazards. The concept of multiple instruction issue was discussed to reduce CPI to less than one. Extensions to Tomasulo's structure by including hardware-based speculation allow speculation that branches are correctly predicted, enabling out-of-order execution but in-order commit after confirming speculation is correct and no exceptions exist.
Limitations of the ILP β Ideal Processor
To understand ILP limitations, an ideal processor is defined as one without artificial constraints on ILP, where the only limits are those imposed by actual data flows through registers or memory. In an ideal processor, all control dependencies and all but true data dependencies are eliminated. Control dependencies are eliminated by assuming perfect branch and jump predictions (all conditional branches and jumps predicted exactly) with perfect speculation and an unbounded instruction buffer. All but true data dependencies are eliminated by assuming infinite virtual registers (facilitating register renaming to avoid WAW and WAR hazards and simultaneous execution of unlimited instructions) and that all memory addresses are known exactly (allowing loads to move before stores if addresses differ).
π Definition β Ideal Processor: A processor without artificial constraints on ILP, where the only limits are those imposed by actual data flows through registers or memory.
Ideal hardware model
By combining these assumptions, in an ideal processor: unlimited instructions (including loads and stores) can issue in one cycle; all functional units have latencies of one cycle so dependent instructions can issue on successive cycles; any instruction in program execution can be scheduled on the cycle immediately following execution of its predecessors; and the last dynamically executed instruction can be scheduled on the very first cycle.
Performance of a Nearly Ideal Processor
The ILP in the Alpha 21264 superscalar processor is examined. Alpha 21264 features: issues up to 4 instructions/cycle, initiates execution on up to 6 instructions, supports large renaming registers (41-integer and 41 floating-point), and uses a large tournament type predictor. Six SPEC92 benchmarks were compiled on a MIPS optimizing compiler. Three floating-point benchmarks are: Fpppp, Doduc, Tomcatv. Three integer programs are: gcc, espresso, li.
The results show that fpppp and tomcatv have extensive parallelism with high instruction-issues, while doduc parallelism doesn't occur in simple loops. The integer program li, a LISP interpreter with many short dependences, offers the lowest parallelism.
The important parameters that limit ILP are: Window Size and Issue Count, Branch and Jump predictors, and Finite number of registers.
Window size and Issue count
In dynamic scheduling, every pending instruction must look at every completing instruction for its operands. A window in an ILP processor is defined as "a set of instructions which is examined for simultaneous execution." The start of the window is the earliest uncompleted instruction and the last instruction determines its size. Since each instruction in the window must be kept in the processor until execution completion, total window size is limited by storage, number of comparisons, and issue rate.
π Formula: Required comparisons per clock cycle = maximum completion rate Γ window size Γ number of operands per instruction
π Example: If maximum completion rate = 6 IPC, window size = 80 instructions, number of operands per instruction = 2 operands, then maximum comparisons required = 6 Γ 80 Γ 2 = 960 comparisons per cycle.
In real processors, the maximum number of instructions that may issue, execute, and commit in the same clock cycle is smaller than the window size. The amount of parallelism uncovered falls sharply with decreasing window size. For benchmark gcc, when window size decreases from 2K to 512, parallelism falls from 35 to 10 IPC, and parallelism reduces to almost zero when window size is 4. Parallelism in integer and FP programs is almost similar for a specific window size.
Branch and jump prediction
The ideal processor assumes perfect branch prediction, but no real processor achieves this. The graph shows five different levels of predictions: Perfect prediction (highest parallelism), Tournament based branch predictor (uses 2-bit correlating and 2-bit non-correlating predictor with a selector; predictor buffer consists of 8K entries each with three 2-bit fields = 48K bits; achieves average accuracy of 97% of six SPEC92 benchmarks), Standard 2-bit predictor with 512 2-bit entries, Profile history of the program, and None (no branch prediction - parallelism limited to within a basic block).
The graph shows extensive difference between programs with loop-level parallelism (tomcatv and fpppp) and those without (integer programs and doduc).
The effect of finite registers
Another important limiting factor is finite registers. The ideal processor eliminates all name dependences assuming infinite physical registers. The Alpha 21264 provides 41 integer and 41 FP extended registers, plus 32 integer and 32 FP architected registers. Having only 32 extra FP and 32 extra GP registers has significant impact on all programs, but the effect is most dramatic on FP programs (fpppp and tomcatv) - instruction-issues increase from 10 to 45 IPC when registers increase from 32 to 128. The reduction in available parallelism is significant when fewer than an unbounded number of renaming registers (less than 32) are available. For integer programs, the impact of having more than 64 registers is not seen due to limitation in window size.
Performance of realizable processors with realistic hardware
A processor with these attributes is considered: up to 64 instructions per clock with no issue restriction, tournament predictor with 1K entries return predictors, perfect memory references done dynamically or through memory dependence predictors, and register renaming with 64 additional integer and 64 additional FP registers. This configuration is more complex and expensive than existing implementations. The results show that the effect of window size for integer programs is not as severe as for FP programs.
Putting it all together
The Intel P6 micro-architecture forms the basis for Pentium Pro, Pentium II, and Pentium III. These processors differ in clock rate, cache architecture, and memory interface. Pentium Pro integrated processor and specialized cache SRAMs into a multichip module. Pentium II uses standard SRAMs as caches. Pentium III has either on-chip 256KB L2 cache or off-chip 512KB cache. The P6 micro-architecture is a dynamically scheduled processor that translates each IA-32 instruction to a series of micro-operations (uops) executed by the pipeline. The maximum number of uops generated per clock cycle is six, with four allocated to the first IA-32 instructions. Uops are executed by an out-of-order speculative pipeline using register renaming and a ROB (Reorder Buffer).
Performance of the Pentium pro implementation
Pentium Pro has the smallest set of primary caches among P6-based microprocessors but has high bandwidth interface to secondary caches.
Branch performance and speculation costs
Branch target addresses are predicted with a 512 entry BTB (Branch Target Buffer). If the BTB does not hit, a static prediction is used. Backward branches are predicted taken (one cycle penalty if correctly predicted). Forward branches are predicted not taken (no penalty if correctly predicted). Branch mispredicts have a direct performance penalty between 10 and 15 cycles, plus indirect penalty from incorrectly speculated instructions. On average, about 20% of branches either miss or are mispredicted and use the simple static predictor rule.
Overall Performance of P6 Pipeline
Overall performance depends on the rate at which instructions complete and commit. On average, one uop commits per cycle. 23% of the time, three uops commit in a cycle. This demonstrates the ability of a dynamically scheduled pipeline to fall behind (55% of cycles, no uops commit) and later catch up (31% of cycles have two or three uops committing).
The Pentium III versus Pentium 4
The Pentium 4 micro-architecture, called NetBurst, is similar to Pentium III's P6 micro-architecture. Both fetch up to three IA-32 instructions per cycle, decode them into micro-ops, and send uops to an out-of-order execution engine executing up to three uops per cycle. However, NetBurst has many differences allowing significantly higher clock rates while maintaining peak to sustained execution throughput.
Differences in Pentium III versus Pentium 4
-
NetBurst has a much deeper pipeline - P6 requires about 10 clock cycles for a simple add instruction from fetch to result availability; NetBurst takes about 20 clock cycles including 2 cycles reserved to drive results across the chip.
-
NetBurst uses register renaming (as in MIPS R10K and Alpha 21264) rather than the reorder buffer used in P6. Register renaming allows up to 128 outstanding results versus 40 permitted in P6.
-
Seven integer execution units in NetBurst versus five in P6, including an additional integer ALU and additional address computation unit. An aggressive ALU operating at twice the clock rate and aggressive data cache lead to lower latencies - basic ALU operations effectively one-half clock cycle in NetBurst versus one in P6; data loads effectively two cycles in NetBurst versus three in P6. These high-speed functional units are critical to lowering potential increase in stalls from the very deep pipeline.
-
NetBurst uses sophisticated trace cache to improve instruction fetch performance, while P6 uses conventional prefetch buffer and instruction cache.
-
NetBurst has a BTB eight times larger with an improved prediction algorithm.
-
NetBurst has 8KB Level-1 data cache compared to P6's 16KB Level-1 data cache, but NetBurst has larger Level-2 cache (256KB) with higher bandwidth.
-
NetBurst implements new SSE2 FP instructions allowing two FP operations per instruction, structured as 12-bit SIMD or short-vector structure, giving Pentium 4 considerable advantage over Pentium III on FP code.
π‘ Why this matters: The trade-offs between pipeline depth, cache size, and execution unit count demonstrate how processor designers balance competing constraints to achieve higher clock rates while maintaining performance throughput.
β Key Takeaways
An ideal processor removes all artificial constraints on ILP, but real processors are fundamentally limited by window size, branch prediction accuracy, and finite registers. The number of comparisons required for dynamic scheduling grows as the product of completion rate, window size, and operands per instruction, creating a practical upper bound on window size. Reducing window size from 2K to 512 can drop parallelism from 35 to 10 IPC for integer programs. Finite registers have the most dramatic effect on floating-point programs, where increasing from 32 to 128 renaming registers can boost IPC from 10 to 45. The Intel P6 micro-architecture demonstrates how these concepts are implemented in real processors, while NetBurst shows that deeper pipelines, register renaming instead of ROB, trace caches, and faster ALUs allow higher clock rates but require careful trade-offs to maintain sustained throughput.
π§ Quick Revision Questions
- What are the four assumptions of an ideal processor for studying ILP limitations?
- What is the formula for calculating comparisons required per clock cycle in a window-based dynamic scheduling processor?
- How does reducing window size from 2K to 512 affect IPC for the gcc benchmark?
- What accuracy does the tournament-based branch predictor achieve on SPEC92 benchmarks, and how many bits does its predictor buffer contain?
- List four key differences between the P6 micro-architecture (Pentium III) and the NetBurst micro-architecture (Pentium 4).