SGI has been working on its next generation shared memory platform, codenamed Ultraviolet, for a long time. The basic idea was to take the NUMAlink-based shared memory of the Altix 4700 line and do that with x86 chips instead of Itaniums. Among other salutary effects, this would reduce the price premium customers have to be prepared to pay for hardware-supported shared memory.
The platform has been in development since well before the most recent bankruptcy and purchase, and the company hasn’t ever publicly wavered on its commitment: in one of CEO Mark Barrenechea’s first interviews following the acquisition, he restated his commitment to the project
“If you don’t believe in UV, you would not have brought the two companies together,” says Barrenechea. “We are fully committed to UV, and it is paramount to our future.”
Today the company announced the realization of that determination: the Altix UV line of x86-based shared memory supercomputers, with first orders shipping to customers who have already signed up in Q2 of 2010.
UV features the fifth generation of the NUMAlink interconnect, offering a 15 GB/sec transfer rate, MPI offload capability in the UV hub chip, and direct access up to 16 TB of shared memory. The system can be configured with up to 2048 Nehalem-EX cores (shipping Q2 next year from Intel) in a single system image, and (as with the 4700) multiple SSIs can be federated together while preserving the single global address space. When I was being briefed on the launch before the show, Jill Matzke, Altix product manager, reminded me that SGI has been very active in the Linux community: all the IP needed to make this shared-memory goodness work has been contributed back to the SUSE and Red Hat communities, so you can actually load a stock distro on your UV when it shows up, and everything will work. If you end up trying that, give me a shout out.
UV will come in two form factors. The smaller Altix UV 100 is a 3U, 19” rackmount unit that scales to 96 sockets (768 cores) and 6TB of shared memory in two racks. Geoffrey Noer, senior director of product marketing for SGI, suggested in our discussions that this could be a good solution for datacenters that want to physically separate their workloads, or for small groups that want the convenience of shared memory without the big upfront commitment.
For those with a little more stretch in their budget, the Altix UV 1000 is a 42U cabinet-level solution that scales to 256 sockets (2048 cores) and 16 TB of memory in four racks. The company also has a notional petascale design, with 256-socket (4-rack) groups in an 8×8 torus. The upper limit on scaling is the UV hub chip, which limits a maximally-configured system to 32,768 sockets in theory today. In both cases the UV compute blade is the same, featuring 8-16 Nehalem-EX cores and up to 128 GB of DDR3 memory. The Nehalems are glued together by Intel’s next-generation Boxboro chipset, which also supports an I/O riser that can be used to add I/O and drives into your cluster (see picture). And because its a single-system image, an I/O riser is available to any blade in the system.
The kinds of applications these systems are aimed at haven’t changed since the value proposition of the shared memory Origins launched way back when: I/O bound and memory intensive apps, very large data problems, graph traversal, and so on. A system like Altix UV could also make a nice fat node for pre- and post-processing tasks (and one of the early customers is using it this way).
Speaking of early customers, SGI is already talking about first customer ships. Noer and Matzke briefed me on four early customers who are scheduled to get their gear in Q2 of next year. The UTenn Center for Remote Data Analysis and Visualization will be using a 1024 core system with 4 TB of memory for visualization of data coming off the university’s big compute systems. HLRN in Germany is buying two systems with a total of 4,352 cores and 18TB memory to go with their already installed 312 TFLOPS ICE system. CALcul en MIdi‐Pyrénées in France is getting a 128 core, 1 TB system, and the University of Hokkaido is getting a 180 core, 360 GB system.
At 1024 and 2048 cores, the Altix 4700 and UV products are far and away the largest SSI’s on the market: the nearest competitor is HP’s Superdome, at 128. But a lot has changed since the Origin days, and SGI’s is no longer the only reasonable solution for getting high performance shared memory. In particular companies like RNA networks, 3-Leaf, and ScaleMP offer less expensive software or software/hardware combination solutions and were first to market with x86_64-based shared memory. They may not scale quite as far as UV, but the biggest chunk of the HPC market share will probably consider these folks “good enough.” Further limiting the potential market for UV is the observation that the class of application designed to run on “average-joe” compute
platforms don’t necessarily need the features and horsepower found in UV. For example, CFD apps whose MPI communication is sped up in the UV hub (for example, barriers are up to 80x faster, and reductions are 2-3x faster according to the company) may not matter much when only using a cluster of 64-128 cores. I say “may not” because we won’t know this for certain until SGI starts publishing performance results, for which they are waiting for the high sign from Intel.
The company is not talking about price either, again waiting for Intel to formally launch the Nehalem-EX chip next year. So while the platform clearly has a lot of potential, without knowing price or performance we really don’t know much.
In case you are wondering about Itanium’s future at SGI, the company hasn’t budged off the party line: the team that briefed me affirmed that the Itanium-based line will remain on offer. I can’t imagine I’ll be saying the same thing this time next year, but as I’ve said many times in this space before, I don’t run a multinational public tech company.
Even for all we don’t yet know, I’m still hopeful for this machine. I’ve been watching and waiting for it for years, and I want to just go touch the thing in their booth this week. This box is the vision of SGI CTO Eng Lim Goh, a gifted architect with a great technology track record. Properly positioned — and priced — this machine still has the potential to be just what at least part of the shared memory HPC crowd needs. And, honestly, SGI needs this system to work. Their non-shared memory platforms mostly compete on price with similar offerings from other manufacturers, and the price premium of the 4700 line has long pushed it out of procurement competitiveness where shared memory wasn’t a driving factor. SGI badly needs a strongly differentiated platform that isn’t priced out of more general procurements in order to build back a healthy revenue stream with a solid margin.