๐ Lecture 42 โ Networks and Clusters (Networks Topology and Internetworking ... Contโd)
๐ Overview: This lecture continues the discussion on network topologies, focusing on centralized switching topologies like Multistage and Omega networks, then moves to distributed switch topologies including Linear Array, Ring, 2D Mesh, Torus, Tree, and Hypercube networks. It also introduces internetworking and clusters, explaining how independent networks communicate reliably.
๐๏ธ Topics Covered
The lecture covers the continuation of switch topologies, including Multistage Interconnect Networks and Omega Network topology with connection rules, Butterfly Network, then transitions to Distributed Switch Networks with performance measure criteria. It details various distributed switch topologies like Linear Array/Ring, Fully Connected, 2D Mesh and Torus, Tree and Fat Tree networks, Hypercube and K-ary n-cube topologies. The lecture concludes with a comparison of network topologies based on bisection bandwidth and cost, followed by brief discussions on internetworking and clusters.
๐ Lecture Summary
Recap: Lecture 41
The recap reviews that a generic interconnection network consists of computer nodes, H/W and S/W interfaces, links, and a communication subnet. Interconnections are classified as LAN, WAN, and SAN based on node count and distance. The communication model uses two unidirectional wires with FIFO queues, and performance is defined by latency as the sum of sender overhead, time to flight, receiver overhead, and message size/bandwidth. Bus-based LANs share a single media but require coordination, while switches provide dedicated lines for faster point-to-point communication. The Crossbar switch is a non-blocking switch using nยฒ switches for n processors, with either source-based or destination-based routing.
Multistage Interconnect Network
Continuing centralized switching topologies, an intermediate class called Multistage network topology lies between crossbar and bus-based networks. It is built from multiple stages of switch boxes, with each stage containing small crossbar switches allowing straight or cross connections. Its performance and cost are more scalable than bus-based networks. The number of identical stages (Nโ) for n nodes with m x m switches is:
๐ Definition โ Multistage Network: A centralized network topology built from multiple stages of crossbar switches to interconnect all nodes, with cost and performance scalable between crossbar and bus-based networks. ๐ Formula: Nโ = logโ n โ The number of stages equals the logarithm base m (switch size) of n (number of nodes). ๐ Formula: Total switches = n/m ร logโ n โ The total number of switches in a multistage network, giving cost O(n log n) compared to O(nยฒ) for crossbar.
Omega Topology: Multistage Interconnect
The Omega Network is a typical multistage network implementation. For 8 nodes addressed with 3-bit code, it uses 3 stages (logโ 8 = 3) of 2x2 crossbar switches, with 4 switches per stage (8/2 = 4). To find the connection pattern, XOR the source and destination addresses. For example, Src (010) โ Dest (110): XOR results in 100, meaning Cross at stage Sโ, Straight at Sโ, Straight at Sโ.
๐ Definition โ Omega Network Connection Rule: For stage i, if the source and destination differ in the iแตสฐ bit, connection is Cross; otherwise, connection is Straight. ๐ Example: Source 010 โ Destination 110. XOR = 100. Since bit 2 differs (0 vs 1), stage Sโ is Cross; bits 1 and 0 are same (1 vs 1, 0 vs 0), so stages Sโ and Sโ are Straight.
Characteristics of Omega
The Omega network is a blocking network because there exists only a single path from source to destination, unlike the non-blocking crossbar. For example, path 010โ110 and path 110โ100 have blockage at Sโ for 110 as it must wait for 010 to pass, otherwise collision occurs. To minimize collisions and improve fault tolerance for high reliability, extra pathways can be added.
Butterfly Network
An alternative to Omega topology is the Butterfly Network. Regardless of source address, for destination aโaโaโ, the iแตสฐ stage switch sends to the upper port if aแตข = 0 and to the lower port if aแตข = 1. ๐ก Why this matters: The Butterfly network provides a different routing strategy compared to Omega, offering alternative path determination based solely on destination bits.
Distributed Switch Networks
Distributed switching networks have switches distributed throughout the network, allowing interconnection of one node to either all nodes or a limited number of nodes. A network where each node interconnects all nodes is called a Fully connected network.
๐ Definition โ Interconnect Performance Measure Criteria:
- Latency: Number of links; should be small.
- Bandwidth: Number or length of messages; should be large.
- Node Degree: Number of links connected to a node.
- Diameter: Maximum distance between any two processors (maximum latency measure).
- Bisect: Imaginary line dividing the interconnect into two equal halves.
- Bisection Bandwidth: Sum of bandwidth of lines crossing the bisection line, measuring communication volume between two halves of the network.
Linear Array / Ring
The simplest distributed switch topology is a Linear Array, where a small switch at every node connects node i to node (i-1) and (i+1) except at endpoints. Messages hop along intermediate nodes until reaching the destination. The Ring network is formed by connecting the 1หขแต and nแตสฐ nodes. A variation called Token Ring uses a single token going around the ring to determine which node can send.
Performance metrics: Cost is O(n) (cheap), overall bandwidth is high, but latency is high at O(N).
๐ Definition โ Token Ring: A ring network variation where a single slot (token) circulates to determine which node is allowed to send a message.
Performance Comparison: Array vs Ring
- Linear Array: Degree = 2, Diameter = N, Bisection width = 1, Bandwidth = N-1, Mean Latency = N/2, Asymmetric, Heterogeneous
- Ring: Degree = 2, Diameter = N/2, Bisection Width = 2, Bandwidth = N, Latency = N/2, Symmetric, Homogeneous
Fully Connected
A Fully connected network is symmetric but expensive, equivalent to crossbar. Every node has a direct link to all other nodes. Performance metrics: Diameter = 1, Degree = n-1, Links = n ร (n-1)/2, Bisects = n ร n/5, Bisection bandwidth proportional to (n/2)ยฒ.
2D Mesh and 2D Torus
2D Mesh or Grid is an asymmetric network where nodes are arranged in an array structure. Each switch has one port for the processor and four ports for nearest-neighbor nodes (left, right, up, down), forming a NEWS communication pattern (North, East, West, South). Connecting unused ports of top/bottom rows and left/right columns with wraparound links forms a 2D Torus.
Performance metrics for n-node 2D Mesh/Torus: Degree = 4, Diameter = 2โN, Bisection width = โN, Bandwidth = N, Asymmetric.
Tree Network Topology
In a Tree Network, switches at each node have ports equal to the number of branches plus one for the processor. For a Binary Tree with two branches per node: Cost is O(N) (cheap), Degree = number of branches (1, 2, 3...), Latency = O(log_deg N), Diameter = 2log_deg N, Bisection Width = 1.
Bottlenecks: The root and branch nodes are bottlenecks. For example, leaf-nodes 1,2 of branch node 9 and 3,4 of branch node 10 can interconnect simultaneously, but leaf-nodes 1,3 and 2,4 cannot due to collision at branch nodes 13, 9, and 10.
Fat Tree Network
To avoid root bottlenecks, multiple paths are provided between nodes in a Fat Tree. Processor-memory nodes connect through multiple stages of crossbars (2ร2, 4ร2+2, 8ร4+4, etc.). This 3D switching increases bandwidth via extra links at each level. The CM-5 uses the Fat Tree concept as a centralized switching network.
Hypercube Network Topology
The Hypercube (binary n-cube) is an n-dimensional interconnect for 2โฟ nodes. For 16 nodes (16 = 2โด, n=4), it's a 4D structure. It requires n ports per switch plus one for the processor, with n nearest neighbor nodes. This minimizes hops with latency O(logโ N).
Performance metrics: Nodes = N = 2โฟ, Degree = n, Diameter = n, Links = n ร 2โฝโฟโปยนโพ, Bisection width = 2โฝโฟโปยนโพ. Other topologies like tree or mesh can be embedded in hypercube. It has good bisection bandwidth but is difficult to layout in 3D space. Popular in early message passing machines like Intel iPSC, NCUBE.
K-ary n-cube Network Topology
The generalization of hypercube is to interconnect k nodes of n-cubes in a string. Total nodes: N = kโฟ. For example, 64 = 4ยณ (4-ary 3-cube). This structure allows wider channels but requires more hops.
Comparing Network Topologies
Comparison for 64-node network:
| Evaluation Category | Bus | Ring | 2D Torus | Fully Connected |
|---|---|---|---|---|
| Performance: Bisection Bandwidth | 1 | 2 | 16 | 1024 |
| Cost: Ports/switch | N/A | 3 | 5 | 64 |
| Total Links | 1 | 128 | 192 | 2080 |
Bus is the standard reference at unit cost; all transfers take time units equal to number of messages. Fully connected has maximum links and ports, all transfers in parallel taking unit time. Ring topology nodes have differing distances.
Internetworking
Internetworking deals with communication of computers on independent and incompatible networks reliably and efficiently. Software standards are enabling technologies, with TCP/IP (Transmission Control Protocol/Internet Protocol) being the most popular internetworking standard. Detailed discussion is beyond this course scope.
๐ Definition โ Internetworking: The process of enabling communication between computers on independent and incompatible networks reliably and efficiently through software standards like TCP/IP.
Cluster
Cluster refers to a group of interconnected computers working together as a single system. The lecture text shows the same definition as internetworking, suggesting clusters utilize internetworking principles for their communication infrastructure.
โญ Key Takeaways
The critical points for exam preparation are: first, understand that Multistage networks provide scalable cost at O(n log n) compared to O(nยฒ) for crossbar, with the Omega network being a blocking network using XOR-based routing where connection is Cross if source and destination bits differ at that stage. Second, master the performance parameters (latency, bandwidth, node degree, diameter, bisection bandwidth) and their values for each distributed topology. Third, remember that Linear Array has diameter N while Ring has diameter N/2 with symmetric structure, 2D Mesh/Torus has degree 4 and diameter 2โN, and Hypercube has degree n and diameter n with 2โฟ nodes. Fourth, know the cost-performance tradeoffs: bus is cheapest but slowest, fully connected is fastest but most expensive, and multistage/torus offer balanced solutions. Finally, recognize that internetworking relies on TCP/IP standards and clusters represent groups of interconnected computers working collectively.
๐ง Quick Revision Questions
- How do you calculate the number of stages and total switches in a Multistage network with n nodes and mรm switches?
- What is the connection rule for Omega Network when routing from source to destination, and why is Omega considered a blocking network?
- What are the six performance measure criteria for distributed switch networks, and what does each measure?
- Compare the diameter, degree, and bisection width of Linear Array, Ring, 2D Torus, and Hypercube for a 64-node network.
- What is the primary internetworking standard mentioned, and what is the purpose of a Fat Tree network?
๐ Lecture 43 โ Networks and Clusters (Internetworks and Clusters)
๐ Overview: This lecture concludes the module on Networks and Clusters by discussing how multiple independent networks are connected together through internetworking, specifically focusing on the TCP/IP protocol suite and the OSI 7-layer model. It then transitions to a detailed study of computer clusters, their design challenges, and four practical cluster design examples, providing a comprehensive understanding of how interconnected systems are built and evaluated in modern computing environments.
๐๏ธ Topics Covered
The lecture begins with a recap of interconnection networks, communication models, software, protocols, and network topologies. It then introduces internetworking, explaining how independent and incompatible networks communicate reliably using protocol families. The OSI 7-layer model and the TCP/IP protocol suite are detailed, followed by an introduction to clusters as System Area Networks. The lecture concludes with four practical cluster design examples comparing uniprocessor, 2-way SMP, and 8-way SMP configurations, including considerations for local disk, storage area networks, other costs, and transaction processing.
๐ Lecture Summary
Recap:
The lecture begins by recapping the previous two lectures on Networks and Clusters. A generic interconnection network comprises Computer nodes, H/W and S/W interface, Links to the interconnection network, and a Communication subnet. The interconnect communication model connects two machines via two unidirectional wires with a FIFO (queue) at the end to hold data. Communication software separates the header and trailer from the message and identifies requests, replies, acknowledgments, and error checking codes. Communication protocols provide the sequence of steps for reliable communication.
The lecture revisits properties and performance of network media (unshielded twisted pair (UTP), coaxial cable, and fiber optics) and the formation of bus-based and switch-based communication subnets. Bus-based subnets share common media where arbitration is the bottleneck, while switch-based networks provide dedicated lines and facilitate faster point-to-point communication. Switch-based networks are classified as centralized and distributed switch networks. Performance of a distributed network is measured by Latency (number of Links between source and destination), Bandwidth (number or length of messages passing per second), Degree (number of links connected to a node), Diameter (number of nodes between source and destination, a measure of maximum latency), Bisection (an imaginary line dividing the interconnect into two equal halves), and Bisection Bandwidth (the volume of communication allowed between any two halves of the network).
The recap also covers the Multistage Switch network, built from large switch boxes each containing small crossbars, whose performance lies between non-locking crossbar and bus-based networks. Distributed-switch interconnects were categorized as fully-connected and partially-connected, symmetric or asymmetric interconnects, with topologies like linear array, ring, 2D mesh/torus, and hypercube studied. The relative cost and performance of these topologies for 64 nodes were compared in a table showing Bisection Bandwidth (Bus: 1, Ring: 2, 2D Torus: 16, Fully Connected: 1024) and Total Links (Bus: 1, Ring: 128, 2D Torus: 192, Fully Connected: 2080).
Internetworking
So far, the discussion focused on interconnection networks. Now, the lecture discusses the connection of two or more interconnection networks, called Internetworking, with the Internet being a typical example. Internetworking deals with the communication of computers on independent and incompatible networks reliably and efficiently. It relies on communication standards to convert information from one kind of network to another. These standards are composed of a hierarchy of layers, where each layer is responsible for a portion of overall communication. Each computer, network, and switch implements its layer of standards, called Protocol Families or Protocol suites, which facilitates applications to work with any inter-connection.
OSI: 7- Layer Model
The Open Systems Interconnect (OSI) developed a 7-layer model describing a network as a series of layers. The Application layer (layer 7) is at the top, and the Physical layer (layer 1) is at the bottom, with Presentation, Session, Transport, Network, and Data Link layers in between.
๐ Definition โ Application Layer (Layer 7): Used for applications specifically written to run over the network, e.g., Network File System (NFS).
๐ Definition โ Presentation Layer (Layer 6): Translates from application to network format and vice versa.
๐ Definition โ Session Layer (Layer 5): Establishes, maintains, and ends the sessions across the network.
๐ Definition โ Transport Layer (Layer 4): Facilitates additional connection below the session layer; the protocol is referred to as the Transmission Control Protocol (TCP).
๐ Definition โ Network Layer (Layer 3): Translates the network address and names to their physical address, e.g., computer name to Media Access Control (MAC); the layer-3 protocol is referred to as Internet Protocol (IP).
๐ Definition โ Data Link Layer (Layer 2): Turns packets into raw bits and, at the receiving end, turns bits into packets; an example protocol is Ethernet.
๐ Definition โ Physical Layer (Layer 1): Transmits raw bit-stream over physical cable/media; IEEE 802 is a typical example physical layer protocol.
TCP/IP Families
The protocol family divides responsibilities among layers, with each layer offering services needed by the layer above. The Transmission Control Protocol/Internet Protocol (TCP/IP) is the most popular internetworking standard and the basis of the Internet.
The protocol at each level is implemented by adding headers and trailers at the sending layer and removing them at the receiving layer. The original message from the top layer includes a header and trailer sent by the lower-level protocol. The next-lower protocol in turn adds its own header (and possibly trailer) to the message, and so on. If the message is too large for a particular layer, it is broken into smaller messages. This division and addition of headers/trailers continues until the message descends to the physical transmission media.
At the receiving end, each level of the protocol family, from bottom to top, checks the message at its level, removes its header and trailer, and passes it to the next higher level. The message is rebuilt by putting the pieces together. This nesting of protocol layers is referred to as the Protocol Stack, reflecting the Last-in First-out nature of addition and removal of headers and trailers.
A typical TCP/IP datagram has standard IP and TCP headers that are 20 bytes each, stacked as shown in the lecture's figure. The length can optionally be increased, specified by the length field (L). The length of the whole datagram is identified by a separate field 'Length' in the IP header, while the TCP header includes this information in the 'sequences number field'.
The lecture notes that detailed discussion on TCP/IP is beyond the scope of this course, and interested students may consult literature on Computer Networks and Internetworking.
Clusters โ System Area Networks
The coordinated use of interconnected computers in a machine room is referred to as a cluster or System Area Network. Massively parallel machines providing high bandwidth can be built from off-the-shelf components instead of custom machines or networks. A cluster, a collection of desktop computers and disks, offers low-cost computing infrastructure that can tackle very large problems and applications such as databases, file servers, Web servers, simulation, and multiprogramming/batch processing.
Clusters face performance confronts: Non-standard connections and Division of memory.
๐ Definition โ Non-Standard Connection Confront: Multiprocessors are usually connected via a memory bus with high bandwidth and low latency, but clusters are connected using the I/O bus of the computer, thus having large conflicts at high speed.
๐ Definition โ Division of Memory Confront: A large single program running on a cluster of N machines requires N independent memory units and N copies of the operating system. In contrast, a shared address multiprocessor allows using almost all memory in the computer.
Despite these challenges, clusters have advantages in dependability and scalability. The weakness of separate memories for program size is a strength in terms of system availability and expandability (scalability). As clusters consist of independent computers connected through LAN, and cluster software runs on top of the local operating system, it is easier than a multiprocessor to replace any computer without bringing down all computers in the cluster. This offers high dependability and scalability, making clusters attractive to worldwide web service providers.
Cluster Design Examples
To study practical aspects of cluster designs, the lecture discusses different cluster designs comprising 32 processors, 32 GB DRAM, and 32 or 64 disks. The designs consider P-III processors operating at 700 MHz and 1000 MHz with L2 cache ranging from 256 KB to 1 MB. Due to larger die size, the processor chip price with 1 MB cache is double that with 256 KB cache, but the objective is to minimize cost for a desired performance target.
Four cases are considered:
- Cost of cluster hardware with local disk
- Cost of cluster hardware with disk over SAN (System or Storage Area Network)
- Cost of cluster options that is more realistic
- Cost and Performance of a cluster for transaction processing
Example 1: Cost of cluster hardware with local disk Three logical organizations are considered: a. Uniprocessor Cluster: 32 xSeries 300 computers (for 32 processors). Maximum memory is 1.5 GB per computer, easily allowing 32 GB (1x32). Each computer has 2 disk drives of 36.4 GB, yielding 32 x 2 x 36.4 = 2330 GB. 32 cables are available for the 1GB Ethernet switch. Since the switch has 30 ports, 2 switches are used, connected together with 4 cables, leaving 56 ports for 32 computers. A standard rack (19" x 30" x 72") accommodates 32 uniprocessor computers and 2 switches (34 rack units). This design is cost-effective.
b. 2-way SMP Cluster: Using 2-processor xSeries 330 computers, everything is halved. 32 processors need only 16 computers. A single 30-port switch works as there are 16 cables. Rack size is 18 RU instead of 44 RU, less than half the standard size.
c. 8-way SMP Cluster: Using 8-processor xSeries 370 computers, only 4 computers are used (4 x 8 = 32 processors). Maximum memory is 32 GB, but only 8 GB per computer is needed, so an 8-port switch is sufficient. At 2 disks per computer, 4 computers can hold 8 disks with maximum capacity of 73.4 GB each, so an expansion storage box (up to 14 disks) and 2 racks are needed.
Comparison of 3-Cluster Designs: The price comparison for 32 processors, 32 GB memory, and 2.3 TB disk shows that network cost decreases as the size of the SMP increases because memory buses supply more inter-processor communication. The 4 of the 8-way SMP cost more than 32 Uniprocessor computers.
Example 2: Cost of cluster hardware using SAN for disks Using local disks reduces cost and space but offers problems: no protection against single disk failure, and state in each computer must be managed separately, resulting in system-down on disk failure. To overcome this, a RAID controller and Fiber Channel Arbitrated Loop (FC-AL) are used as the storage area network (SAN). All SCSI disks are replaced with FC-AL disks behind the RAID storage server. FC-AL can be connected in a loop with up to 127 devices. The price comparison shows that SAN cost also shrinks as servers increase in the number of processors per computer.
Example 3: Cluster design considering other costs The first two designs considered hardware cost only, but software (database) cost and hardware maintenance cost (operator cost) were not considered. Other costs include backup tapes and space to house servers. A complete comparison shows that 2-way SMP using SAN is lowest in total price. Notably, over 3 years, the cost of the operator will be more than the cost of hardware, so the purchase cost of old computers must be reduced to reduce overall cost.
Example 4: Cluster design for transaction processing Using 32 P-III processors with the same IBM computer building block, key differences are:
- Disk Size: Small and fast disks are used as this structure cares more about I/Os per second (IOPS).
- RAID: No RAID is required as the performance benchmark doesn't include human cost.
- Memory: Maximum DRAM is packed into servers, so each of the four 8-way SMPs has 32 GB, yielding 128 GB.
- Processor: 900 MHz P-III with 2 MB L2 cache is used.
The cost-performance analysis shows that almost half of the cost is in software, installation, and maintenance. The lecture concludes that as the purchase cost is less than half the cost of ownership, the cost of hardware solves only a part of the problem.
Summary
The module on Networks and Clusters covered the formation of generic interconnection networks comprising computer nodes (host or end system), H/W and S/W interface, links to the interconnection network, and communication subnet. Interconnections are designated as Local Area Network (LAN), Wide Area Network (WAN), and System (or Storage) Area Network (SAN).
The interconnect communication model shows two machines connected via two unidirectional wires with a FIFO (queue) at the end. Communication software separates header and trailer from the message. Communication protocols suggest steps for reliable communication. Network performance defines message latency as the sum of sender overhead, time to flight, receiver overhead, and the ratio of message size to bandwidth.
The Multistage Switch lies between crossbar and bus-based networks. The number of identical stages (Ns) of large switch boxes, each having m x m crossbar switches, in a network with n nodes, is equal to logm n, and the switches per stage is n/m. The cost of a multistage switch network is O(n log n), considerably smaller than a crossbar network's O(n2) for large n. Examples are Omega and Butterfly networks.
Distributed-switch interconnect topologies include linear array, ring, 2D mesh/torus, and hypercube. The lecture today discussed internetworking, i.e., the connection of two or more interconnection networks to communicate reliably and efficiently. Internetworking relies on communication standards composed of a hierarchy of layers. The Transmission Control Protocol/Internet Protocol - TCP/IP is the most popular internetworking standard. The protocol at each level is implemented by adding headers and trailers at the sending layer and removing at the receiving layer.
After introducing internetworking, the lecture discussed computer clusters, which are the coordinated use of interconnected computers in a machine room. Clusters face challenges of Non-standard connections and Division of memory, but have advantages in dependability and scalability. The practical aspects of cluster designs were studied through four examples.
โญ Key Takeaways
Internetworking connects independent, incompatible networks reliably through layered communication standards like the OSI 7-layer model and TCP/IP protocol suite, where each layer adds or removes headers/trailers in a protocol stack. Clusters are cost-effective, scalable, and dependable systems built from off-the-shelf components connected via I/O buses, though they face challenges from non-standard connections and divided memory. In cluster design, network cost decreases as SMP size increases because memory buses handle more inter-processor communication, but larger SMPs can be more expensive overall. For transaction processing, maximizing IOPS with small, fast disks and packing maximum DRAM is critical, while software and maintenance costs often exceed hardware costs over a system's lifetime. The multistage switch network offers a cost-efficient O(n log n) interconnect solution that balances performance between crossbar and bus-based networks.
๐ง Quick Revision Questions
- What are the four components of a generic interconnection network?
- What is internetworking, and why does it rely on communication standards composed of a hierarchy of layers?
- What are the two main performance confronts (challenges) of clusters compared to multiprocessors?
- In the cluster design examples, why does network cost decrease as the size of the SMP increases?
- What key insight does the fourth cluster design example (for transaction processing) reveal about the total cost of ownership of a cluster?
๐ Lecture 44 โ Putting It All Together (Case Studies)
๐ Overview: This lecture serves as a capstone, analyzing three real-world advanced computer architectures: the PowerPC 750, the PowerPC 970 FX, and the Intel Pentium VI (P6). By studying these specific implementations, all the theoretical concepts of superscalar, out-of-order, and speculative execution are brought together to show how they work in practice and how different design choices impact performance.
๐๏ธ Topics Covered
This lecture provides detailed case studies of three advanced microarchitectures: the PowerPC 750, a 32-bit RISC superscalar processor with six execution units and out-of-order completion; the PowerPC 970 FX, a 64-bit deeply pipelined implementation with aggressive branch prediction and out-of-order execution; and the Intel Pentium VI (P6), which introduced Intel's dynamic execution micro-architecture with an in-order fetch/decode, out-of-order dispatch/execute, and in-order retire unit.
๐ Lecture Summary
PowerPC 750 - General
The PowerPC 750 is a high-performance implementation of the 32-bit portion of the PowerPC architecture, a reduced instruction set computer (RISC) microprocessor. It provides 32-bit effective addresses for integer data types (8, 16, and 32 bits) and floating-point data types (32 and 64 bits). Its superscalar architecture features six execution units and two register files, allowing it to fetch up to four instructions per cycle from the instruction cache, dispatch as many as two instructions per clock, and execute up to six instructions per clock.
PowerPC Instructions are encoded as single-word (32-bit) with consistent formats across all instruction types, permitting efficient decoding in parallel with operand accesses. This fixed instruction length greatly simplifies instruction pipelining. Instructions are categorized as: Integer (arithmetic, compare, logical, rotate, shift), Floating-point (arithmetic, multiply/add, rounding, conversion, compare, status/control), Load/store (integer and floating-point loads/stores, atomic memory operations like lwarx and stwcx), Flow control (branching, condition register logical, trap), Processor control (synchronizing memory accesses, cache, TLB, segment register management), and Memory control (control of caches, TLBs, and SRs).
๐ก Why this matters: The consistent instruction format is a hallmark of RISC design, directly enabling the parallel decode and pipelining that give the 750 its high performance.
PowerPC 750 โ Instruction Flow
The instruction flow in PowerPC 750 includes three stages: Instruction fetch, Instruction decode, and Instruction dispatch. The architecture allows a maximum of four instructions to be fetched per clock cycle. The number of clock cycles necessary to request instructions from the memory system depends on where the target instruction is located: the branch target instruction cache (BTIC), the on-chip instruction L1 cache, or the L2 cache.
In the decode/dispatch stage, instructions can only be dispatched from the two lowest instruction queue entries, IQ0 and IQ1. A maximum of two instructions can be dispatched per clock cycle, although an additional branch instruction can be handled by the Branch Processing Unit (BPU). Only one instruction can be dispatched to each execution unit per clock cycle. For a dispatch to occur, there must be: a vacancy in the specified execution unit, a rename register available for each destination operand, and an open position in the completion queue. If no entry is available in the completion queue, the instruction remains in the instruction queue (IQ).
PowerPC 750 โ Execution Units
The PowerPC 750 superscalar pipeline contains two integer units (IUs). IU1 can execute any integer instruction, while IU2 can execute all integer instructions except multiply and divide. These share thirty-two General Purpose Registers (GPRs) and a single-entry reservation station for each. It also features a three-stage floating-point unit (FPU) that supports both single- and double-precision operations with hardware support for denormalized numbers and a single-entry reservation station, using thirty-two 64-bit FPRs. The two-stage Load/Store Unit (LSU) contains a two-entry reservation station, provides single-cycle pipelined cache access, and has a three-entry store queue. It supports both big- and little-endian modes and has a dedicated adder for effective address (EA) calculations. It also performs alignment and precision conversion for floating-point data and sign extension for integer data.
PowerPC 750: Completion Unit
The Completion unit retires an instruction from the six-entry reorder buffer (completion queue) when three conditions are met: all instructions ahead of it have been completed, the instruction has finished execution, and no exceptions are pending. The completion unit guarantees the sequential programming model (precise exception model). It monitors all dispatched instructions, retires them in order, tracks unresolved branches, flushes instructions from mispredicted branches, and can retire as many as two instructions per clock.
PowerPC 750 Rename Buffers
The 750 provides rename registers to hold instruction results before the completion unit commits them to the architected register. There are six GPR rename registers, six FPR rename registers, and one each for the Condition Register (CR), Link Register (LR), and Count Register (CTR). When an instruction is dispatched, a rename register for its results is assigned, and the dispatcher provides a tag to the execution unit identifying the rename register that will forward required data. Results are transferred from the rename registers to the architected registers by the completion unit when an instruction is retired. Results of squashed instructions are flushed from the rename registers.
PowerPC 750 Branch Prediction Unit
The Branch Prediction Unit supports both static and dynamic branch predictions, only one of which is used at any given time. Static branch prediction is defined by the PowerPC architecture, using the BO field in branch instructions to allow software to hint whether a branch is likely to be taken. The 750 uses this encoding to predict the branch direction before the condition is known. Dynamic branch prediction uses a 512-entry Branch History Table (BHT) with two bits per entry, allowing predictions of: Not-taken, Strongly not-taken, Taken, Strongly taken.
PowerPC 750 Branch Target Cache - BTC
The 750 uses the Branch Target Instruction Cache (BTIC) to reduce the time required for fetching target instructions when a branch is predicted to be taken. The BTIC is a 64-entry (16-set, four-way set-associative) cache of branch instructions. When a BTIC hit occurs, instructions are fetched into the instruction queue a cycle sooner than they could be made available from the instruction cache.
PowerPC 750 Multiple Branch Prediction
The 750 executes through two levels of prediction. Instructions from the first unresolved branch can execute, but they cannot complete until the branch is resolved. If a second branch instruction is encountered in the predicted instruction stream, it can be predicted, and instructions can be fetched (but not executed) from the second branch. No action can be taken for a third branch instruction until at least one of the two previous branch instructions is resolved.
PowerPC 750 Cache
The 750 has separate on-chip instruction and data caches. The instruction cache is 32-Kbyte, eight-way set-associative using a Pseudo Least-Recently-Used (PLRU) replacement policy. Both caches have 32-byte (eight-word) cache blocks, are physically indexed with physical tags, and support per-block cache write-back or write-through operation. The caches can be disabled or locked in software. Data cache coherency (MEI) is maintained in hardware. The critical double word is made available to the requesting unit first, and the cache is non-blocking.
The Data Cache is organized with 32-word blocks (5 bits), 128 sets (7 bits), Tags (20 bits), and uses the MEI state (Modified, Exclusive, Invalid). The Instruction Cache has a similar structure (32-word blocks, 128 sets, 20-bit tags) but uses a Valid/Not valid state and is not snooped.
PowerPC 750: Multiprocessing
The 750's multiprocessing support features a hardware-enforced, three-state cache coherency protocol (MEI) for the data cache and a load/store with reservation instruction pair for atomic memory references and semaphores. The MEI protocol supports the Modified, Exclusive, and Invalid states. Notably, there is no shared state in this protocol.
๐ก Why this matters: The lack of a 'Shared' state in the MEI protocol is a simpler, less expensive design choice compared to protocols like MESI, but it may lead to more cache-to-cache transfers in a multiprocessor system.
PowerPC 970 FX
The PowerPC 970 FX is a 64-bit implementation of the PowerPCยฎ AS Architecture (version 2.01) that includes Vector/SIMD Multimedia eXtension. It is a deeply pipelined design with varying pipeline depths: 16 stages for most fixed-point register-register operations, 18 stages for most load/store operations (assuming an L1 D-cache hit), up to 25 stages for floating-point operations, and 19 stages for vector permute operations. A key feature is dynamic instruction cracking, where some complex instructions are broken into two simpler, more RISC-like instructions, allowing for a simpler inner core dataflow.
PowerPC 970 FX: General
This processor features aggressive branch prediction, capable of predicting up to two branches per cycle and supporting up to 16 predicted branches in flight, including both branch direction and branch addresses. It has in-order dispatch of up to five operations into a distributed issue queue structure and out-of-order issue of up to 10 operations into 10 execution pipelines: two load/store, two fixed-point, two floating-point, one branch, one condition register, one vector permute, and one vector ALU operation. It employs register renaming and uses the MERSI (Modified/Exclusive/Recent/Shared/Invalid) cache coherency protocol. It supports a theoretical maximum of 215 instructions in flight, distributed across various buffers and queues. It features fast, selective flush of incorrect speculative instructions and results, with a specific focus on storage latency management, including out-of-order and speculative issue of loads, support for up to eight outstanding L1 cache line misses, hardware-initiated instruction prefetching from L2 cache, and software-initiated data stream prefetching.
PowerPC 970 FX: Instruction Fetch
The instruction fetch unit uses a 64KB, direct-mapped instruction cache (I-cache) with 128-byte lines broken into four 32-byte sectors and a dedicated 32-byte read/write interface from the L2 cache using a critical sector first reload policy. It has a four-entry, 128-byte, instruction prefetch queue above the I-cache with hardware-initiated prefetches. It can fetch a 32-byte aligned block of eight instructions per cycle.
PowerPC 970 FX: Branch Prediction
The branch predictor can scan all eight fetched instructions for branches each cycle and predict up to two branches per cycle. It uses a three-table prediction structure: a local predictor (16K entries, 1-bit each, Taken/Not taken), a global predictor (16K entries, 1-bit each, with an 11-bit history XORed with the branch instruction address), and a selector (16K entries, 1-bit each, indexed similarly, choosing between the local and global predictor). This combination produces very accurate predictions. It also has a 16-entry link stack for address prediction of subroutine returns (with stack recovery) and a 32-entry count cache for predicting the target address of bcctr instructions.
PowerPC 970 FX: Instruction Decode and Preprocessing
This unit has a three-cycle pipeline to decode and preprocess instructions, including cracking one instruction into two internal operations. Cracked and micro-coded instructions have access to renamed emulation registers: eGPRs (4), eFPR (1), and eCR field (1), in addition to architected facilities. It includes an 8-entry (16 bytes per entry) instruction fetch buffer that can take up to eight instructions in and output five instructions out each cycle.
PowerPC 970 FX: Instruction Dispatch and Completion Control
The dispatch unit has four dispatch buffers that can hold up to four dispatch groups when the global completion table (GCT) is full. It uses a 20-entry global completion table for group-oriented tracking, associating a five-operation dispatch group with a single GCT entry. This tracks internal operations from dispatch to completion for up to 100 operations and supports precise exceptions. It is capable of very fast restoration for instructions on group boundaries (e.g., branches) and slower restoration for instructions contained within a group.
PowerPC 970 FX: Branch and Condition Register Execution Pipeline
There is one branch execution pipeline that computes the actual branch address and direction for comparison with the prediction. If prediction was incorrect, it redirects instruction fetching and assists in training the branch table predictors, link stack, and count cache. There is one condition register logical pipeline that executes CR logical instructions, CR movement operations, and some mtspr/mfspr instructions. It uses out-of-order issue with a bias towards the oldest operations first.
PowerPC 970 FX: Data Stream Prefetch
The 970 FX supports eight (modeable) data prefetch streams in hardware. All eight are available if vector prefetch instructions are disabled. If vector prefetch is enabled, four vector prefetch streams are supported using four of the eight hardware streams. The vector prefetch mapping algorithm supports the most commonly used forms of vector prefetch instructions.
Intel P-VI: General
The Intel P6 family of processors succeeds the Pentiumยฎ line and implements Intel's dynamic execution micro-architecture, which combines multiple branch prediction, data flow analysis, and speculative execution.
Intel P-VI: Major Units
The P6 architecture has three main engines and a bus interface unit. The FETCH/DECODE unit is an in-order unit that takes the user program instruction stream from the instruction cache, decodes them into a series of ฮผ-operations (ฮผops), representing the dataflow. The pre-fetch is speculative. The DISPATCH/EXECUTE unit is an out-of-order unit that accepts the dataflow stream, schedules execution of the ฮผops subject to data dependencies and resource availability, and temporarily stores the speculative results. The RETIRE unit is an in-order unit that knows how and when to commit (or "retire") the temporary, speculative results to the permanent architectural state. The BUS INTERFACE unit communicates directly with the L2 cache, supporting up to four concurrent cache accesses and controlling a transaction bus with the MESI snooping protocol to system memory.
Intel P-VI: Inside Fetch
The L1 Instruction Cache fetches the cache line corresponding to the Next_IP and presents 16 aligned bytes to the decoder. The decoder converts Intel Architecture instructions into triadic ฮผops (two logical sources, one logical destination). Most instructions are converted into single ฮผops, some into one-to-four ฮผops, and complex instructions require microcode. The ฮผops are queued and sent to the Register Alias Table (RAT), where logical register references are converted into physical register references. The ฮผops are then entered into the instruction pool, which is implemented as a Content Addressable Memory called the Re-Order Buffer (ROB).
Intel P-VI: Inside Dispatch/Execute
The Dispatch unit selects ฮผops from the instruction pool based on their status. If a ฮผop has all its operands and the required execution resource is available, the Reservation Station removes the ฮผop and sends it for execution. The results are later returned to the pool. There are five ports on the Reservation Station, allowing a peak rate of 5 ฮผops per clock, though a sustained rate of 3 is more typical. The Branch Target Buffer (BTB) correctly predicts most branches. If a branch is mispredicted, the Jump Execution Unit (JEU) changes the status of all ฮผops behind the branch to remove them from the instruction pool, and the proper branch destination is provided to the BTB to restart the pipeline.
Intel P-VI: Inside Retire
The Retire Unit checks the status of ฮผops in the instruction pool. To retire a ฮผop, it must re-impose the original program order. It reads the instruction pool to find candidates for retirement and determines which are next in the original program order. It then writes the results to the Retirement Register File (RRF). The Retire Unit is capable of retiring 3 ฮผops per clock.
Intel P-VI: Bus Interface Unit
Loads are encoded into a single ฮผop, while stores require two ฮผops: one to generate the address and one to generate the data. These ฮผops must later re-combine for the store to complete. Stores are never performed speculatively because there is no transparent way to undo them. Stores are also never re-ordered among themselves. A store is dispatched only when both the address and data are available and there are no older stores awaiting dispatch. A study of memory access reordering concluded that: constraining stores from passing other stores has a small performance impact; constraining stores from passing loads has an inconsequential loss; and constraining loads from passing other loads or stores has a significant impact on performance. The Memory Order Buffer (MOB) allows loads to pass other loads and stores, acting like a reservation station and re-order buffer for suspended loads and stores.
โญ Key Takeaways
The three case studies demonstrate different design philosophies for achieving high performance. The PowerPC 750 is a classic RISC superscalar design with in-order dispatch and out-of-order completion, relying on a straightforward 6-stage pipeline and simple branch prediction. In contrast, the PowerPC 970 FX is a deeply pipelined, heavily out-of-order design with sophisticated features like dynamic instruction cracking, a three-table branch predictor, and a large instruction window of up to 215 instructions. The Intel P6 introduces a radically different "dynamic execution" micro-architecture with a clear separation of concerns: an in-order fetch/decode unit that cracks CISC instructions into RISC-like ฮผops, a central out-of-order execution engine, and an in-order retire unit to ensure precise exceptions. A critical design lesson from the P6 is the importance of load-load and load-store reordering for performance, while store-store and store-load ordering can be more constrained without significant loss.
๐ง Quick Revision Questions
- What are the three conditions that must be met for an instruction to be dispatched in the PowerPC 750?
- How does the PowerPC 970 FX's three-table branch prediction structure (local, global, selector) improve prediction accuracy over the simpler BHT used in the 750?
- In the Intel P6 architecture, why can't stores be performed speculatively, and how does this constraint affect the Memory Order Buffer (MOB)?
- What is "dynamic instruction cracking" in the PowerPC 970 FX, and what is its primary benefit?
- What is the key difference in the cache coherency protocols between the PowerPC 750 (MEI) and the PowerPC 970 FX (MERSI)? What does the extra state ('Recent') imply?