SGI Origin 3000 and Onyx 3000

• Origin 3200C - This model was a cluster of nodes that consist of entire Origin 3200 systems. This model could scale to thousands of processors. The clustering technology used was gigabit Ethernet and Infiniband.

Physically, the Origin 3000 is based on "bricks" - rackmounted modules that provide a specific function, that are connected together using NUMAlink 3 cables for modules providing compute functions, or Crosstown2 cables for modules providing I/O functions. These bricks are mounted in a standard 19-inch rack. There are two racks for the Origin 3000, a 17U-high short rack, and a 39U-high tall rack.

Architecturally, the Origin 3000 is based on the distributed shared memory NUMAflex architecture. The NUMAlink 3 system interconnect uses a fat tree hypercube network topology.

The C-brick is a 3U-high enclosure that contains a node on a PCB. The node contains two or four processors, the local and directory memory, and the Bedrock ASIC. It connects to the system using NUMAlink 3.

The two processors and their secondary caches is contained on a PIMM (Processor Integrated Memory Module) daughter card that plugs into two 240-pin connectors on the node board. Initially, the Origin 3000 used the 360 MHz R12000 and the 400 MHz R12000A processors with 4 or 8 MB of secondary cache. In May 2001, the 500 MHz R14000 was introduced with 8 MB of secondary cache and in February 2002, the 600 MHz R14000A was made available. Near the end of its lifetime, the C-brick was updated with 800 MHz MIPS processors.

The C-Brick supports 512 MB to 8 GB of local memory through eight DIMM slots organised into eight banks by using proprietary 100 MHz ECC DDR SDRAM DIMMs with capacities of 256 MB, 512 MB and 1 GB. The data path between the DIMM and the Bedrock ASIC is 144 bits wide, with 128 bits for data and 16 bits for ECC. Because the Origin 3000 uses a distributed shared memory model, directory memory is used to maintain cache coherency between the processors. Unlike the Origin 2000, which requires dedicated proprietary DIMMs for the directory memory, the Origin 3000's directory memory is integrated in the same DIMMs that contain the local memory. Due to this, there are two kinds of DIMM used in the Origin 3000: standard DIMMs, which supports systems with up to 128 processors, and premium DIMMs, which supports systems with more than 128 processors. The 256 MB DIMM is a standard DIMM, the 1 GB DIMM is a premium DIMM and the 512 MB DIMM can be either.

The Bedrock ASIC connects the processors, local and directory memory and the Crosstown2 interface to the NUMAlink 3 system interconnect using a crossbar. The ASIC contains six major sections: the crossbar (XB), two processor interfaces (PI_0 and PI_1), the memory and directory interface (MD), the I/O interface, (II) and the network interface (NI). The interfaces communicate with each other via FIFO buffers that are connected to the crossbar. It also serves as the memory controller. Although each PIMM contains two microprocessors, but only has one 1.6 GB/s interface to the Bedrock ASIC, the single interface is multiplexed to enable the two processors on each PIMM to operate independently without bus contention at the cost of reduced bandwidth.

The CX-brick is a 4U-high enclosure that is only used in Origin 3900 and Onyx 3900 systems. It differs from the C-brick by containing four node boards and eight-port router ASIC. The CX-brick can support up to 16 processors and 32 GB of memory. The CX-brick initially used the IP53 motherboard that supported 500 MHz R14000 and 600 MHz R14000A processors with 8 MB secondary caches, later upgraded to use the R16000 and R16000A. It connects to the system using NUMAlink 3.

The R-brick is a 2U-high enclosure that features an eight-port router ASIC. Its purpose is to route NUMAlink packets throughout the system to connect the C-Bricks together. R-bricks for the Origin 3400 have a router ASIC with two ports disabled to prevent them from being upgraded into Origin 3800 systems.

The I-brick is a 4U-high enclosure that provides boot I/O functions for the Origin 3000. It features five hot swappable PCI-X slots, with three clocked at 33 MHz and two at 66 MHz on two separate buses, two sled-mounted 3.5-inch Fibre Channel hard drives and a proprietary CD-ROM drive. The I-Brick also provides a 10/100BASE-T Ethernet port, an IEEE 1394 port, a serial port, two USB ports as well as a real-time clock and NVRAM for storing configuration information through the IO9. It connects to the system using Crosstown2.

The IX-Brick is a 4U-high enclosure that is only used in Origin 3900 and Onyx 3900 systems. It an updated version of the I-brick with 133 MHz PCI-X expansion slots. It connects to the system using Crosstown2 cables.

The P-brick is a 4U-high enclosure that provides an additional 12 PCI-X expansion slots on six buses to the system. It connects to the system using Crosstown2 cables.

The PX-brick is a 4U-high enclosure that is only used in Origin 3900 and Onyx 3900 systems. It an updated version of the P-brick with 133 MHz PCI-X expansion slots. It connects to the system using Crosstown2 cables.

The X-brick is a 4U-high enclosure that provides four XIO expansion slots. It connects to the system using Crosstown2 cables.

The D-brick is a 4U-high enclosure that can support 12 hot-swappable Fibre Channel hard drives through two Fibre Channel loops.

The G-brick is a 18U-high enclosure that supports the InfiniteReality graphics subsystem. Each G-brick can support two InfiniteReality3 or InfiniteReality4 graphics pipes, although only one pipe can have four raster manager boards while the other can have two. The G-brick connects to the system using Crosstown2 cables.

The V-brick is a 4U-high enclosure that supports two InfinitePerformance graphics pipes. Each graphics pipe consists of a 128 MB SGI VPro V12 graphics card. The V-brick connects to the system using Crosstown2 cables.

The N-brick is a 2U-high enclosure that allows the C-bricks to connect to the G-bricks without using X-bricks or I-bricks, thus saving space as the other bricks are taller. The N-brick was intended for configurations where I/O capabilities were not required.