π Lecture 20 β Instruction Level Parallelism (Static Scheduling)
π Overview: This lecture transitions from hardware-based dynamic scheduling to software-based static scheduling techniques for exploiting Instruction Level Parallelism (ILP). It covers basic compiler scheduling, loop unrolling and scheduling, and static branch prediction, explaining how these methods reduce data and control hazards to improve pipeline performance.
ποΈ Topics Covered
This lecture begins with a recap of dynamic scheduling techniques studied in previous lectures, then introduces static scheduling as a software approach to exploit ILP. It covers basic compiler pipeline scheduling to reduce data hazard stalls, explains loop unrolling with and without scheduling using a vector addition example, discusses limits to loop unrolling gains such as code size growth and register pressure, explores loop unrolling and scheduling in superscalar processors with multiple instruction issues per cycle, and concludes with static branch prediction methods and their accuracy rates.
π Lecture Summary
Recap: Dynamic Scheduling
Our discussions in the last eight lectures focused on hardware-based approaches to exploit parallelism among instructions. The instructions in a basic block, a straight-line code sequence without branches, are executed in parallel using a pipelined datapath. The performance of pipelined datapath is limited by its structure and data and control dependences, as they lead to structural, data, and control hazards. These hazards are removed by introducing stalls, which degrade performance by increasing the CPI to more than 1. The number of stalls is reduced or eliminated by introducing additional hardware and using dynamic scheduling techniques. Key hardware techniques include forwarding and bypassing (reducing data hazard stalls), delayed branching and branch scheduling (reducing control hazard stalls), basic dynamic scheduling with scoreboarding (reducing data hazard stalls from true dependences), dynamic scheduling with renaming using Tomasulo's approach (reducing stalls from anti-dependences and output dependences), dynamic branch prediction, speculation, and multiple instruction issues per cycle.
Introduction to Static Scheduling in ILP
Multiple-instruction-issue per cycle processors exist as superscalar processors, VLIW processors, and vector processors. Superscalar processors exploit ILP using both static and dynamic scheduling approaches, while VLIW processors use only static scheduling. The major software scheduling techniques under discussion include basic compiler scheduling (reducing data hazard stalls), loop unrolling (reducing control hazard stalls), compiler dependence (reducing ideal CPI and data hazard stalls), trace scheduling (reducing ideal CPI and data hazard stalls), and compiler speculation (reducing ideal CPI and data and control hazard stalls).
Basic Pipeline Scheduling
To exploit ILP, we must keep a pipeline full with a sequence of unrelated instructions that can be overlapped. A dependent instruction must be separated from the source instruction by a distance equal to the latency of that instruction. For example, an FP ALU operation using the result of an earlier FP ALU operation must be kept 3 cycles away, and an FP ALU operation using the result of an earlier load double word operation must be kept 1 cycle away. The assumed latencies are: Integer ALU operation latency = 0, FP Load latency to FP store = 0 (result bypassed without stalling), Integer Load latency = 1, and FP ALU operation latency to FP store = 2. A compiler performing scheduling must consider the latencies of functional units in the pipeline.
π Definition β Latency: The number of intervening clock cycles needed to avoid a stall between a producing instruction and a consuming instruction. π Functional Unit Latencies: Integer ALU (0), FP Load to FP Store (0), Integer Load (1), FP ALU to FP Store (2).
Execution of a Simple Loop with Basic Scheduling
Consider a simple loop: for (i=1000; i>0; i=i-1) x[i] = x[i] + scalar, where a scalar is added to a vector in 1000 iterations, and the body of each iteration is independent at compile time. The MIPS code without scheduling is:
L.D F0, 0(R1) ;F0 array element
ADD.D F4, F0, F2 ;add scalar in F2
S.D F4, 0(R1) ;store result
DADDU R1, R1, #-8 ;decrement pointer 8 bytes
BNE R1, R2, LOOP ;branch R1! =R2
Data dependencies in ADD and STORE operations lead to data hazards, and the BNE instruction causes a control hazard.
π Example: Single Loop Execution Without Scheduling Using a standard five-stage pipeline with branch delay of one clock cycle and the given latencies:
Instructions: Clock cycles:
L.D F0, 0(R1) 1
Stall 2 (L.D followed by FP ALU op has latency=1)
ADD.D F4, F0, F2 3
Stall 4 (FP ALU op followed by STORE has latency=2)
Stall 5
S.D F4, 0(R1) 6
DADDUI R1, R1, #-8 7
Stall 8 (Double ALU has latency=1)
BNE R1, R2, LOOP 9
Stall 10 (Branch has latency=1)
This code requires 10 clock cycles per iteration.
π Example: Single Loop Execution With Compiler Scheduling
Loop: Clock cycles:
L.D F0, 0(R1) 1
DADDUI R1, R1, #-8 2
ADD.D F4, F0, F2 3
Stall 4
BNE R1, R2, LOOP 5 (delayed branch)
S.D F4, 8(R1) 6 (altered & interchanged with DADDUI)
To schedule the delayed branch, the compiler determined it could swap DADDUI and S.D by changing the destination address of S.D from 0(R1) to 8(R1) because R1 has been decremented by DADDUI. This loop requires 6 clock cycles per iteration. Actual work (load, add, store) takes 3 cycles, and the remaining 3 cycles are loop overhead (condition evaluation, stall, branch), resulting in 100% overhead.
π‘ Why this matters: Compiler scheduling reduces execution time from 10 to 6 cycles per iteration, a 40% improvement, by rearranging independent instructions to fill pipeline stalls.
Loop Unrolling
To eliminate or reduce the impact of loop overhead (3 clock cycles per loop), we need more operations within the loop relative to overhead instructions. A simple way is to replicate the loop body for multiple iterations and adjust the loop termination code. This approach is known as loop unrolling.
π Example: Loop Unrolling Without Scheduling Unroll the loop to have 4 copies of the body. Assume R1 is initially a multiple of 32, and registers are not reused:
LOOP:
L.D F0, 0(R1)
ADD.D F4, F0, F2
S.D F4, 0(R1) ;drop ADDUI & BNE
L.D F6, -8(R1)
ADD.D F8, F6, F2
S.D F8, -8(R1) ;drop ADDUI & BNE
L.D F10, -16(R1)
ADD.D F12, F10, F2
S.D F12, -16(R1) ;drop ADDUI & BNE
L.D F14, -24(R1)
ADD.D F16, F14, F2
S.D F16, -24(R1)
DADDUI R1, R1, #-32
BNE R1, R2, LOOP
Without scheduling, this loop executes in 28 clock cycles (14 for instruction issue, 7 stalls). This is slower than the scheduled version which completes one iteration in 6 cycles, so 24 cycles for 4 iterations.
π Example: Unrolling with Scheduling The unrolled loop is scheduled assuming the same latencies:
Loop: Clock cycles:
L.D F0, 0(R1) 1
L.D F6, -8(R1) 2
L.D F10, -16(R1) 3
L.D F14, -24(R1) 4
ADD.D F4, F0, F2 5
ADD.D F8, F6, F2 6
ADD.D F12, F10, F2 7
ADD.D F16, F14, F2 8
S.D F4, 0(R1) 9
S.D F8, -8(R1) 10
DADDUI R1, R1, #-32 11
S.D F12, -16(R1) 12
BNE R1, R2, LOOP 13
S.D F16, -24(R1) 14
The 4 LOAD instructions are executed sequentially without stalls. The STORE of iteration 1 is issued in clock cycle 5 without stall. DADDUI is issued in cycle 11 to avoid stall before BNE (cycle 13). STORE of iteration 4 is issued in cycle 14, eliminating stall after branch. Execution time drops to 14 cycles total, or 3.5 clocks per element, compared to 7 cycles per iteration before scheduling.
Loop Unrolling and Scheduling: Conclusion
The key to performing loop unrolling and scheduling is knowing when and how to change instruction ordering. Decisions must: determine legality of moving S.D after DADDUI and BNE and adjust offset; determine loop iterations are independent except for maintenance code; use different registers to avoid unnecessary constraints; eliminate extra test and branch instructions and adjust termination code; determine that loads and stores from different iterations are independent; and schedule code preserving dependencies.
Limits to the Gains of Loop Unrolling and Scheduling
- Loop Overhead: After unrolling, overhead was only 2 cycles (DADDUI and BNE), reduced to 2/4 or Β½ cycles per iteration. This could be reduced to 2/8 or ΒΌ cycles with 8 times unrolling.
- Growth in Code Size: For larger loops, code size growth may decrease cache miss rate.
- Register Pressure: Aggressive unrolling may result in shortfall of registers, called Register Pressure, making it impossible to allocate registers to live values. The compiler must take care of these limitations.
π Definition β Register Pressure: The shortfall of registers that occurs after aggressive unrolling and scheduling, when it may not be possible to allocate registers to live values.
Loop Unrolling and Scheduling with Multiple Issues in Superscalar
Consider the same loop example with two issues per cycle (one load/store/branch or integer ALU and one FP). The loop is unrolled 5 times. After unrolling, the loop contains 5 L.D, ADD.D, S.D, and one each DADDUI and BNE:
Integer instruction: FP instruction: Clock cycle:
Loop: LD F0,0(R1) 1
LD F6,-8(R1) 2
LD F10,-16(R1) ADDD F4,F0,F2 3
LD F14,-24(R1) ADDD F8,F6,F2 4
LD F18,-32(R1) ADDD F12,F10,F2 5
SD 0(R1),F4 ADDD F16,F14,F2 6
SD -8(R1),F8 ADDD F20,F18,F2 7
SD -16(R1),F12 8
SD -24(R1),F16 9
SUBI R1,R1,#40 10
BNEZ R1,LOOP 11
SD -32(R1),F20 12
The unrolled superscalar loop runs in 12 clocks, or 2.4 clocks per iteration, compared to 14/4 = 3.5 for scheduled and unrolled on simple 5-stage MIPS pipeline, exhibiting an improvement factor of 1.5.
Static Branch Prediction
Dynamic branch predictors predict branches based on run-time behavior. Delayed Branch was introduced as a static branch prediction technique that exposes a pipeline hazard so the compiler can reduce penalty. Static branch predictors are used when branch behavior is highly predictable at compile time, which is helpful for scheduling data hazards like loop unrolling.
π Example: Static Branch Prediction Consider an example from conditional selection branches:
LD R1, 0(R2)
DSUBU R1, R1, R3
BEQZ R1, L
OR R4, R5, R6
DADDU R10, R4, R3
L: DADDU R7, R8, R9
The dependence of DSUBU and BEQZ on L.D shows a stall is needed after L.D. If the branch (BEQZ) is predicted almost always taken, and the value of R7 is not needed on the fall-through path, then moving L: DADDU R7, R8, R9 after the L.D improves speed. If the branch is rarely taken, and R4 is not needed on the taken path, moving the OR instruction after L.D could be considered. Scheduling depends on knowing branch behavior.
Methods to statically predict branch behavior: A. Predict a branch as taken: Average misprediction rate equals untaken branch frequency (34% for SPEC), ranging from 59% to 9%. B. Predict on basis of branch direction: Choose backward going branches as taken and forward going as not taken. For SPEC, more than half of forward branches are taken, so predicting all as taken is better. C. Predict on basis of profile information: More accurate, using profile data from earlier runs. Branch behavior is often bi-modal (highly biased toward taken or not taken).
For FP benchmarks on SPEC 92, misprediction rate varies from 4% to 9%, and for integer programs from 5% to 15%.
β Key Takeaways
This lecture teaches that static scheduling is a software approach to exploit ILP where the compiler rearranges instructions to reduce pipeline stalls. Basic compiler scheduling reduces data hazard stalls by separating dependent instructions by their latency distance. Loop unrolling replicates the loop body multiple times to reduce loop overhead and increase parallelism, and when combined with scheduling, can dramatically reduce execution time from 10 cycles per iteration to 3.5 cycles. The key limits to loop unrolling are loop overhead, code size growth, and register pressure. Multiple instruction issue in superscalar processors, combined with loop unrolling and scheduling, can further improve performance by a factor of 1.5. Finally, static branch prediction methods (predict taken, predict by direction, or use profile information) achieve misprediction rates of 4% to 15%, significantly enhancing superscalar processor performance.
π§ Quick Revision Questions
- What are the three main limits to the gains of loop unrolling and scheduling, and how does each affect performance?
- For the single loop example, compare the clock cycles per iteration for unscheduled execution, scheduled execution, unrolled without scheduling, and unrolled with scheduling. What improvement factor does scheduling provide?
- What is the difference between register pressure and code size growth as limits to loop unrolling?
- How does static branch prediction using profile information differ from the simple "predict taken" strategy, and what are the typical misprediction rates for each?
- In the superscalar loop example with two issues per cycle, why is the loop unrolled 5 times instead of 4, and what is the final improvement factor over the simple pipelined scheduled version?
π Lecture 21 β Instruction Level Parallelism (Static Scheduling β Multiple Issue Processor)
π Overview: This lecture extends the discussion of static scheduling to VLIW (Very Long Instruction Word) processors. It covers how compilers schedule multiple instructions in VLIW architectures, including loop unrolling for VLIW, detection of loop-carried dependences, and advanced compiler techniques like the GCD test and tree-height reduction to enhance instruction-level parallelism.
ποΈ Topics Covered
The lecture begins with a recap of static scheduling and branch prediction, then introduces static multiple issue using the VLIW approach. It covers VLIW/EPIC processor architecture and compares overhead with superscalar processors. A detailed example demonstrates VLIW loop unrolling with performance analysis. The lecture then shifts to detecting and enhancing loop-level parallelism, covering loop-carried dependence, the GCD test for dependence detection, and techniques like back substitution and tree-height reduction to eliminate dependent computations.
π Lecture Summary
Recap: Static Scheduling and Branch Prediction
In superscalar processors, inserting stalls is the basic compiler approach to avoid data and control hazards. The compiler schedules instructions to reduce or eliminate stalls. For loops, unrolling enhances performance, and scheduling the unrolled loop by repeating each instruction for multiple iterations further reduces stalls, though additional registers are needed. In superscalar processors with multiple issues, static branch prediction results in a lower misprediction rate (4% to 15%) compared to dynamic branch prediction.
Static Multiple Issue: VLIW Approach
A VLIW (Very Long Instruction Word) processor contains a fixed set of instructions (typically 4-16 instructions per word). A VLIW is formatted either as one large instruction or a fixed instruction packet with explicit parallelism among instructions. VLIW is also referred to as EPIC (Explicitly Parallel Instruction Computing) because explicit parallelism exists among instructions.
π Definition β VLIW processor: A processor that initiates multiple instructions per cycle by having the compiler pack operations into a wide template or packet (64-128 bytes), and the hardware simply issues the complete packet.
π‘ Why this matters: In VLIW, the overhead does not grow with issue-width, unlike superscalar processors where overhead grows with issue-width. Early VLIW machines required recompilation for different hardware versions, but recent innovations eliminate this need.
Example: VLIW Processor with Loop Unrolling
Consider a VLIW processor that can perform up to five operations per cycle: one integer operation, two floating-point operations, and two memory reference operations. Instructions have 16-bit to 24-bit fields per unit, with instruction length ranging from 112 to 168 bits.
For the MIPS loop x[i] = x[i] + s:
Loop: L.D F0, 0(R1) ;load array element
ADD.D F4, F0, F2 ;add scalar in F2
S.D F4, 0(R1) ;store result
DADDU R1, R1, #-8 ;decrement pointer 8 bytes
BNE R1, R2, LOOP ;branch if R1 != R2
Assumptions: Compiler uses local scheduling; processor has sufficient registers (15 vs. 6 in superscalar); loop is unrolled 7 times to eliminate all stalls.
VLIW Schedule (9 cycles for 7 iterations):
Clock 1: LD F0,0(R1) | LD F6,-8(R1)
Clock 2: LD F10,-16(R1) | LD F14,-24(R1)
Clock 3: LD F18,-32(R1) | LD F22,-40(R1) | ADDD F4,F0,F2 | ADDD F8,F6,F2
Clock 4: LD F26,-48(R1) | (empty) | ADDD F12,F10,F2 | ADDD F16,F14,F2
Clock 5: (empty) | (empty) | ADDD F20,F18,F2 | ADDD F24,F22,F2
Clock 6: SD 0(R1),F4 | SD -8(R1),F8 | ADDD F28,F26,F2
Clock 7: SD -16(R1),F12 | SD -24(R1),F16
Clock 8: SD -32(R1),F20 | SD -40(R1),F24 | SUBI R1,R1,#48
Clock 9: SD -0(R1),F28 | BNEZ R1,LOOP
π Definition β Issue rate: The average number of operations issued per cycle.
π Formula: Average issue rate = Total instructions issued / Total cycles β Plain meaning: How many operations the processor completes on average each clock cycle.
π Example: 23 instructions (7Γ3 loop instructions + 1 condition evaluation + 1 branch) issued in 9 cycles = 23/9 = 2.5 operations per cycle (compared to 3.5 for superscalar).
π Formula: VLIW efficiency = (Operations issued) / (Total available slots) = 23/45 = 0.51 or 51%
π Example: Each VLIW instruction has 5 slots, so 9 instructions provide 45 total slots; only 23 contain operations.
π Formula: Cycles per iteration = Total cycles / Number of iterations = 9/7 = 1.29 cycles per iteration
π‘ Why this matters: VLIW code runs twice as fast as superscalar scheduled code for this example, despite lower efficiency.
Detecting and Enhancing Loop Level Parallelism
LLP (Loop Level Parallelism) emphasizes determining the type of dependence among operands across loop iterations, while ILP (Instruction Level Parallelism) focuses on dependence among instructions within a loop.
Loop Carried Dependence (LCD) occurs when data accessed in a later iteration depends on the data value produced in a previous iteration.
π Definition β Loop Carried Dependence (LCD): A data dependence where the value computed in one iteration is used in a later iteration of the same loop.
Example 1 β LLP without LCD:
For (i=1000; i>0; i=i-1)
x[i] = x[i] + s;
Here, dependence exists between two uses of x[i] within the same iteration (not loop-carried), but successive uses of i in different iterations is loop-carried. This can be eliminated by loop unrolling.
Example 2 β LCD Preventing ILP:
For (i=1; i<=100; i=i+1) {
A[i+1] = A[i] + C[i]; // S1
B[i+1] = B[i] + A[i+1]; // S2
}
- First dependence (loop-carried): S1 uses A[i] computed in previous iteration; S2 uses B[i] computed in previous iteration
- Second dependence (not loop-carried): S2 uses A[i+1] computed by S1 in the same iteration
Example 3 β LCD not preventing parallelism:
For (i=1; i<=100; i=i+1) {
A[i] = A[i] + B[i]; // S1
B[i+1] = C[i] + D[i]; // S2
}
S1 depends on S2 (uses B[i] from previous iteration), but S2 does not depend on S1. No cycle exists, so the loop can be parallelized by reordering statements.
Reorganized code:
A[1] = A[1] + B[1];
for (i=1; i<99; i=i+1) {
B[i+1] = C[i] + D[i];
A[i+1] = A[i+1] + B[i+1];
}
B[101] = C[100] + D[100];
Finding Dependences: Greatest Common Divisor (GCD) Test
An affine index can be written as a Γ i + b, where a and b are constants and i is the loop index variable.
π Definition β Affine index: An array index of the form aΓi+b, where a and b are constants.
Two references to the same array have a dependence if:
- Two iteration indices j and k are both within loop bounds (m β€ j β€ n, m β€ k β€ n)
- The values are equal: aΓj + b = cΓk + d
π Formula: GCD Test β If a loop-carried dependence exists, then GCD(c, a) must divide (d-b)
π Example: Check for dependence in for (i=1; i<=100; i=i+1) { X[2*i+3] = X[2*i] * 5.0; }
- For X[2*i+3]: a=2, b=3
- For X[2*i]: c=2, d=0
- GCD(a,c) = 2
- d-b = 0-3 = -3
- 2 does NOT divide -3, so no dependence exists
β οΈ Limitation: The GCD test is sufficient to guarantee no dependence, but there are cases where GCD test succeeds but no dependence exists (because GCD test does not consider loop bounds).
Classifying Dependences and Enhancing ILP
Compilers classify dependence types to recognize name dependences and eliminate them through renaming and copying.
Back Substitution increases parallelism by replacing dependent computations with equivalent expressions, though it may increase the total amount of computation.
Tree-Height Reduction increases parallelism by reorganizing operations using associativity, potentially increasing the number of operations while reducing the critical path.
π Definition β Tree-Height Reduction: An optimization that reorganizes a sequence of dependent operations into a tree structure that can be evaluated in fewer steps using associativity.
π Example: Original code (3 cycles)
ADD R1,R2,R3
ADD R4,R1,R6
ADD R8,R1,R7
Transformed using associativity (2 cycles):
ADD R1,R2,R3
ADD R4,R6,R7
ADD R8,R1,R4
The first two ADD instructions can be issued in parallel.
π Example: Recurrence sum = sum + x
Unrolled 5 times with values x1 through x5:
- Original: sum = sum + x1 + x2 + x3 + x4 + x5 (5 dependent operations)
- Optimized: sum = ((sum + x1) + (x2 + x3)) + (x4 + x5) (3 dependent operations)
β Key Takeaways
VLIW processors achieve static multiple-issue by having the compiler pack operations into wide instruction packets, eliminating the hardware overhead that grows with issue-width in superscalar processors. Loop unrolling is essential for VLIW to fill all operation slots, and the average issue rate (2.5 for the example) may be lower than superscalar (3.5), but VLIW achieves fewer cycles per iteration (1.29 vs. 2.67), making it twice as fast. Loop-carried dependence (LCD) is the primary obstacle to loop-level parallelism, where data from one iteration is needed in a later iteration; compilers detect these using the GCD test on affine indices. Compiler techniques like back substitution and tree-height reduction can eliminate or reduce dependent computations, transforming sequential code into more parallel forms, even at the cost of increasing the total number of operations.
π§ Quick Revision Questions
- What is the key architectural difference between VLIW and superscalar processors in terms of who schedules instructions?
- In the VLIW loop unrolling example, why were the ADD.D operations scheduled in cycle 3 instead of cycle 2?
- What does the GCD test determine, and what is its limitation regarding loop bounds?
- In the example
A[i] = A[i] + B[i]; B[i+1] = C[i] + D[i];β why is this loop parallelizable despite the loop-carried dependence? - How does tree-height reduction increase parallelism, and what is the trade-off in terms of total operations?
π Lecture 22 β Instruction Level Parallelism (Software Pipelining and Trace Scheduling)
π Overview: This lecture explores advanced compiler techniques for enhancing Instruction Level Parallelism beyond basic loop unrolling. It covers software pipelining as a space-efficient alternative to loop unrolling, and global code scheduling methods (trace scheduling and superblocks) for handling loops with internal control flow. These techniques are critical for maximizing performance in VLIW and superscalar processors.
ποΈ Topics Covered
The lecture begins with a recap of eliminating dependent computations through copy propagation and tree-height reduction, then introduces software pipelining as a technique that interleaves instructions from different iterations without full loop unrolling. It explains symbolic loop unrolling and compares software pipelining with traditional loop unrolling. The second half covers global code scheduling complexities, trace scheduling with its two-step process of trace selection and compaction, and superblocks as an improvement over traces by allowing multiple exits but only a single entry point.
π Lecture Summary
Recap: Lecture 21
Last time, the discussion covered Static Scheduling for VLIW (Very Long Instruction Word) processors, where a VLIW is formatted as one large instruction or a fixed instruction packet with explicit parallelism among instructions in a set. Multiple operations are initiated in a cycle by the compiler which places them in a packet. Wider processors with multiple independent functional units are used to eliminate recompilation.
The compiler finds dependence and schedules instructions for parallel execution, resulting in improvement over superscalar processors in:
- β Average issue rate (operations issued per cycle)
- β Execution speed (time to complete execution of code)
However, VLIW efficiency (percentage of available slots containing an operation) ranges from 40% to 60%. The lecture distinguished between ILP (Instruction Level Parallelism) and LLP (Loop Level Parallelism), noting that loop-carried dependence prevents LLP but not ILP. The affine-based GCD (Greatest Common Divisor) algorithm detects dependence in loops.
Reducing Dependent Computations
To achieve more ILP, the compiler reduces dependent computations using Back Substitution technique, resulting in algebraic simplification and optimization that eliminates operations which copy values to simplify the sequence.
Copy Propagation
The approach of simplification and optimization is also called Copy Propagation. For example, in the sequence:
DADDUI R1,R2,#4
DADDUI R1,R1,#4
The net use of R1 is to hold the result of the second DADDUI operation. Substituting the result of the first DADDUI operation into the second results in:
DADDUI R1,R2,#8
This eliminates the multiple use of register R1 during loop unrolling.
π Definition β Copy Propagation: A compiler optimization technique where the result of one operation is substituted directly into subsequent operations to eliminate redundant computations and simplify code sequences.
Conclusion: Particularly in memory access, this technique eliminates multiple increments of array indices during loop unrolling and moves increments across memory addresses.
Tree-Height Reduction β Optimization
The copy-propagation technique reduces the number of operations or code length. However, optimization to increase parallelism of the code is possible by restructuring the code.
Tree-Height Reduction (Restructuring) increases the number of operations while reducing execution cycles. This optimization is called tree-height reduction because it reduces the height of the tree structure representing a computation, making it wider but shorter.
π Formula: Original sequence (3 cycles):
ADD R1,R2,R3
ADD R4,R1,R6
ADD R8,R1,R7
π Example: Using associativity, the code can be transformed to:
ADD R1,R2,R3
ADD R4,R6,R7
ADD R8,R1,R4
This sequence executes in 2 cycles by issuing the first two instructions in parallel, compared to 3 cycles for the original.
π‘ Why this matters: Tree-height reduction trades off increased operation count for decreased execution time, exploiting associativity of operations to create more parallel execution opportunities.
Conclusion: Detecting and Enhancing LLP
The analysis of LLP focuses on determining data dependence of some later iteration on an earlier iteration. This dependence is referred to as Loop-Carried Dependence. The GCD test detects existence of dependence. Compiler techniques such as Copy Propagation and Tree-Height Reduction eliminate dependent computations.
Uncovering Instruction Level Parallelism
Loop Unrolling is the basic compiler technique to uncover ILP. Loop unrolling with compiler scheduling enhances the overall performance of Superscalar and VLIW processors. It generates a sequence of straight-line code uncovering parallelism among instructions. To avoid pipeline stalls, dependent instructions are separated from source instructions by a distance in clock cycles equal to the pipeline latency of that source instruction. The compiler determines:
- β The amount of ILP available in the program
- β The latencies of the functional units in the pipeline
Advanced Compiler Techniques
Two new compiler techniques to uncover ILP are discussed:
- Software pipelining
- Global Code Scheduling (Trace scheduling and Superblock)
Software Pipelining
Software pipelining is a technique where the loop is reorganized such that the code for each iteration is made by choosing instructions from different iterations of the original loop. A software-pipelined loop interleaves instructions from different iterations without unrolling the loop.
The instructions are chosen from different loop iterations, separating the dependent instructions within one iteration of the original loop.
π Example Code: Adding constant F2 to array R1:
Loop: L.D F0,0(R1)
ADD.D F4,F0,F2
S.D 0(R1),F4
DADDUI R1,R1,#-8
BNE R1,R2, LOOP
For software pipelining, the compiler symbolically unrolls the loop and schedules them. It selects instructions from each iteration that do not have dependence among each other. The overhead instructions (DADDUI and BNE) are not replicated.
Symbolic Loop Unrolling for three iterations shows instructions selected from each iteration (shown in yellow in the original):
- Iteration i: L.D F0,0(R1) | ADD.D F4,F0,F2 | S.D F4,0(R1)
- Iteration i+1: L.D F0,0(R1) | ADD.D F4,F0,F2 | S.D F4,0(R1)
- Iteration i+2: L.D F0,0(R1) | ADD.D F4,F0,F2 | S.D F4,0(R1)
Resulting Software Pipeline:
Loop: S.D F4,16(R1) ; stores in M[i]
ADD.D F4,F0,F2 ; adds to M[i-1]
L.D F0,0(R1) ; loads M[i-2]
DADDUI R1,R1,#-8
BNE R1,R2, Loop
For start-up and finish-up, additional code is needed. The loop takes 5 clock cycles to execute per result. L.D and ADD.D are separated by offset of 16 to run the loop for three iterations (i.e., two iterations less than simple loop unrolling and scheduling). Registers F4, F0, and R1 are reused as there is no data dependence, thus WAR (Write-After-Read) hazard is avoided.
Software Pipelining vs. Loop Unrolling
Software pipelining can be thought of as symbolic loop unrolling because some algorithms use loop unrolling techniques to software-pipeline loops.
Key differences:
- β Software pipelining consumes less code space compared to loop unrolling
- β Software pipelining reduces overhead (fill & drain pipe) to once per loop at the beginning and end, versus once per each unrolled iteration in loop unrolling
For example, if a loop is to do 100 iterations with 4 iterations symbolically unrolled, overhead is paid for 100/4 = 25 times instead of 100 times with loop unrolling.
Global Code Scheduling
Loop unrolling and code scheduling work well when the loop body is straight-line code. Software pipelining works well when the body is a single basic block. Global Code is one where the loop body has internal control flow, such as if-then-else conditional blocks.
Effective scheduling of such code is complex since it involves moving instructions across branches into the shortest possible sequence to compact a code fragment. Global code involves both data dependence and control dependence.
Commonly used compiler-based approaches to schedule global code:
- β Trace scheduling (critical path approach)
- β Superblock approach
Complexities of Global Code Scheduling
Consider a typical global code fragment representing an iteration of an unrolled inner loop. Moving assignments to B or C prior to condition evaluation requires complex analysis.
Code sequence example:
LD R4, 0(R1) ; Load A
LD R4, 0(R1) ; Load B
DADDU R4, R4, R5 ; A+B
SD R4, 0(R1) ; A=A+B
BNEA R4, Else_part ; Is A=0?
: : ; THEN part
SD ...., 0(R2) ; store to B
J Join ; jump to F
Else_part:
x ..... ; x code
Join:
SD ...., 0(R3) ; store c[i]
Effect of moving assignment to B before BNEA: If B is referenced in X or after the IF statement, moving B before IF will change the data flow. This can be overcome by making a shadow copy of B before IF and using the shadow copy in x. However, such a copy is avoided as it slows down the program.
Moving assignment to C before the first branch is more complex and requires two steps:
- Step 1: Assignment is moved over the join point of the ELSE part into the THEN part. This movement makes instructions for C control-dependent - they are not executed if the ELSE path is chosen. To ensure correct execution, a copy of the instruction is made in the ELSE part.
- Step 2: If C assignment is moved to before the IF Test, provided it does not affect any data flow, the copy of instruction in the ELSE part can be avoided.
π‘ Why this matters: Global code scheduling is extremely complex because moving instructions across branches can change both data flow and exception behavior. Compilers must carefully manage compensation code to ensure correctness.
Trace Scheduling
Trace is a sequence of basic blocks whose operations could be put together into smaller numbers of instructions. Trace scheduling is a way to organize the global code motion process such that the cost of code motion is incurred by the less frequent paths.
π Definition β Trace Scheduling: A compiler technique for scheduling global code by focusing on the most frequently executed path (trace), moving operations early in the sequence, and using compensation code for less frequent paths.
Use cases: Useful for processors with a large number of issues per clock where:
- β Conditional or predicted execution is inappropriate or unsupported
- β Simple loop unrolling is not sufficient to uncover ILP to keep the processor busy
Two steps of Trace Scheduling:
- Trace Selection: Find likely sequence of basic blocks whose operations could be put together into smaller numbers of instructions
- Trace Compaction: Squeeze the trace into a small number of wide instructions
Trace Generation: Since the probability of loop branches-taken is usually high, trace is generated by loop unrolling. Additionally, static branch prediction is employed as taken or not-taken to obtain straight-line code by concatenating many basic blocks.
In the earlier global code fragment, if the true path is taken as most frequent, the primary trace could be generated by unrolling the true path n- (say 4-) times. Here, Trace Exit is jump-off the frequent path and Trace Enter is return to trace.
Code Compaction is the code scheduling where the compiler attempts to:
- β Move operations as early as it can in a sequence (trace)
- β Pack the operations into as few wide instructions (or issue packets) as possible
Advantages of Trace Scheduling:
- β Simplifies the decision concerning global code motion
- β Branches are viewed as jumps into (Trace Entrance) or jump out of (Trace Exit) the selected trace which is the most probable path
Overhead of Trace Scheduling:
- When code is moved across a trace, additional bookkeeping code is needed on entry or exit points
- When an entry or exit point is in the middle of a trace, significant overheads of compensation code may make trace scheduling unattractive
Superblocks
The drawback of Trace scheduling is that entries into and exit out of a trace in the middle cause significant complications, and the compiler requires to generate and track compensation code.
π Definition β Superblocks: A form of extended basic blocks that have a single entry point but allow multiple exits. This makes compaction easier compared to traces.
Superblocks overcome the trace scheduling drawback. In the earlier global code example, superblocks with one entrance can be constructed by moving C appropriately.
Tail Duplication is used to create a separate block that corresponds to the portion of the trace after the entry. Each unrolling of the loop creates an exit from the superblock to the residual loop that handles the remaining iterations. The residual loop handles the iterations that occur when the unpredicted path is selected.
Advantages of Superblocks:
- β Reduces the cost of bookkeeping and scheduling compared to general trace generation
- β However, code size may enlarge more than a trace-based approach
- β Like trace scheduling, superblocks scheduling may be most appropriate when other techniques fail
Conclusion β Enhancing ILP
All four approachesβLoop Unrolling, Software Pipelining, Trace Scheduling, and Superblocksβaim to increase the amount of ILP that can be exploited by a processor issuing more than one instruction on every clock cycle.
Numerical Problems
Example 1: Find dependences in the loop:
For (i=2; i<100; i= i+1) {
a[i] = b[i] + a[i]; /* s1 */
c[i-1] = a[i] + d[i]; /* s2 */
a[i-1] = 2 * b[i]; /* s3 */
b[i+1] = 2 * b[i]; /* s4 */
}
Six dependences identified:
- Antidependence from s1 to s1 on
a - True dependence from s2 to s1 (value of
ain s2 depends on result ofain s1) - Loop-carried true dependence from s4 to s1 on
b - Loop-carried true dependence from s4 to s3 on
b - Loop-carried true dependence from s3 to s3 on
b - Loop-carried output dependence from s3 to s3 on
a
True dependences 3, 4, and 5 are loop-carried and cannot be removed by renaming. These factors imply the loop is not parallel as written. The loop can be made parallel by rewriting to find a functionally equivalent loop.
Example 2: Dot product loop with recurrence (running sum in F2 initially 0):
| Instruction producing result | Instruction using result | Latency |
|---|---|---|
| FP ALU op | Another FP ALU op | 3 |
| FP ALU op | Store double | 2 |
| Load double | FP ALU op | 1 |
| Load double | Store double | 0 |
Foo:
L.D F0,0(R1) ; load X[i]
L.D F4,0(R2) ; load Y[i]
MUL.D F0,F0,F4 ; X[i]*Y[i]
ADD.D F2,F0,F2 ; sum = sum + X[i]*Y[i]
DADDUI R1,R1,#-8 ; decrement X index
DADDUI R2,R2,#-8 ; decrement Y index
BNEZ R1,foo ; loop if not done
Part a): Unrolling twice results in:
foo:
L.D F0,0(R1)
L.D F4,0(R2)
L.D F6,#-8(R1)
MUL.D F0,F0,F4 ; 1 from L.D F4,0(R2)
L.D F8,#-8(R2)
DADDUI R1,R1,#-16
MUL.D F6,F6,F8 ; 1 from L.D F8,-8(R2)
ADD.D F2,F0,F2 ; 3 from MUL.D F0,F0,F4
DADDUI R2,R2,#-16
Stall
BNEZ R1,foo
ADD.D F2,F6,F2 ; in slot, 3 from ADD.D F2,F0,F2
The dependence chain from one ADD.D to the next forces a stall.
To eliminate the stall: Use commutativity and associativity of dot product with two running sums (one for even elements, one for odd elements):
foo:
L.D F0,0(R1)
L.D F6,-8(R1)
L.D F4,0(R2)
L.D F8,-8(R2)
MUL.D F0,F0,F4 ; 1 from L.D F4,0(R2)
MUL.D F6,F6,F8 ; 1 from L.D F8,-8(R2)
DADDUI R1,R1,#-16
DADDUI R2,R2,#-16
ADD.D F2,F0,F2 ; 3 from MUL.D F0,F0,F4
BNEZ R1,foo
ADD.D F2,F0,F2 ; 3 from MUL.D F6,F6,F8, fill delay slot
ADD.D F2,F0,F2 ; combine even and odd elements
Part b): The unrolled and scheduled code (loop body takes 10 cycles):
| Integer Inst | FP Inst | Clock Cycles |
|---|---|---|
| L.D F0,0(R1) | 1 | |
| L.D F6,-8(R1) | 2 | |
| L.D F4,0(R2) | 3 | |
| L.D F8,-8(R2) | 4 | |
| DADDUI R1,R1,#-16 | MUL.D F0,F0,F4 | 5 |
| DADDUI R2,R2,#-16 | MUL.D F6,F6,F8 | 6 |
| stall | 7 | |
| stall | 8 | |
| BNEZ R1,foo | ADD.D F2,F0,F2 | 9 |
| ADD.D F2,F0,F2 | 10 | |
| Bar: | ADD.D F2,F0,F2 | 14 |
Example 3: GCD test for loop dependence
For (i=2; i<=100; i+=2)
a[i] = a[50*i+1]
Normalized version (start index at 1, increment by 1):
For (i=1; i<=50; i++) {
a[2*i] = a[(100*i)+1]
}
GCD Test: a=2, b=0; c=100, d=1
- gcd(2,100) = 2
- d - b = 1
- Since 1 is a factor of 2, GCD test indicates there IS a dependence
Reality: There is no actual dependence because the loop loads values from a[101], a[201]...a[5001] and stores to a[2], a[4]...a[100]. The GCD test gives a false positive in this case.
β Key Takeaways
The four compiler techniquesβloop unrolling, software pipelining, trace scheduling, and superblocksβall aim to increase the amount of ILP that can be exploited by processors issuing multiple instructions per cycle. Software pipelining is more code-space efficient than loop unrolling because it interleaves instructions from different iterations without replicating the entire loop body, paying the fill/drain overhead only once per loop rather than once per iteration. Global code scheduling addresses the challenge of loops with internal control flow, where trace scheduling focuses on the most frequent execution path and uses compensation code for less frequent paths, while superblocks improve upon traces by allowing only a single entry point with multiple exits, reducing bookkeeping complexity. The GCD test is a useful but imperfect tool for detecting loop-carried dependencesβit can yield false positives when the actual array indices accessed never overlap. In dot product reduction problems, exploiting associativity and commutativity to compute partial sums (e.g., even and odd elements separately) can eliminate pipeline stalls that would otherwise be forced by the recurrence.
π§ Quick Revision Questions
-
What is the key difference between software pipelining and traditional loop unrolling in terms of code space and overhead?
-
Explain the two steps of trace scheduling and describe why compensation code may be needed when moving instructions across trace boundaries.
-
How do superblocks differ from traces, and what advantage does the single-entry point provide?
-
In the dot product example, why does computing two partial sums (even and odd elements) eliminate the stall caused by the recurrence?
-
Given the GCD test result (d-b mod gcd(c,a) = 0 indicates dependence), explain why the test can produce false positives and why this matters for compiler optimization.