SeaMicro Drops 64-bit Atom Bomb Server

By Timothy Prickett Morgan • Get more from this author

If SeaMicro has been “awash in business” as company founder, Andrew Feldman, tells El Reg it has been in the wake of last June’s launch of the 512-core, 10U Atom-based SM10000 server, then the company had better batten down the hatches and prepare for the deluge. Because today SeaMicro is shipping the second generation of its ultra-dense integrated server/switch/load balancing platform based on a 64-bit version of Intel’s Atom processors.

As El Reg reported last summer when SeaMicro came out of stealth mode, the company had put together a very clever and compact server based on 32-bit, single-core Atom Z530 processors. The SM10000 put eight of these Atom Z530 processors and their chipsets on a single card, with SeaMicro’s homegrown ASICs for creating a proprietary chassis backplane linking the server nodes to in-chassis Gigabit and 10 Gigabit Ethernet switches in the back of the chassis, disk drives and controllers in the front of the chassis, and a load balancer for managing traffic across the networks and fabric lashing it all together.

Cramming 512 cores of any kind into a 10U space along with storage and networking to support those nodes is not an easy task, no matter what chip architecture you are using. But Gary Lauterbach, a chip designer who used to work for Advanced Micro Devices who also designed the former Sun Microsystems UltraSparc-III and UltraSparc-IV processors, put together a team of engineers from AMD, Cisco Systems, Force10 Networks, Juniper Networks, and Sun Microsystems to get it done. Feldman, who used to run marketing at Force10, is SeaMicro’s chief executive officer and is an evangelist for compact systems that can do hyperscale work, but do so more efficiently than standard x64-based machines.

SeaMicro was founded in July 2007, and the original SM10000 server board and its system took more than three years to develop, plus $25m in venture funding from Khosla Ventures, Draper Fisher Jurvetson, Crosslink Capital, and an unnamed private backer. SeaMicro also cadged a $9.3m slice of a $47m grant in January 2010 from the US Department of Energy to come up with some greener technologies for the data center.

The first SeaMicro system board had eight of the single-core Atom Z530 Atom and their “Poulsbo” US15W chipsets soldered onto it, thus:

The first-generation SeaMicro Atom Z530 server board

What SeaMicro developed – and what makes the SM10000 server unique – is an ASIC chip that virtualizes disk access and Ethernet networking for each of the Atom servers. The ASIC also implements a 3D torus interconnect between all of the 64 server boards nodes in the chassis, which delivers 1.28 Tb/sec of aggregate bandwidth across the cores and boards in the box. The SM10000 also has a little something called Dynamic Compute Allocation Technology, or DCAT, which is a home-grown field programmable gate array (FPGA) that does load balancing across the cores and boards.

The load balancing electronics are hooked into the SM10000’s system management tools to allow for pools of servers to be grouped together and managed as a single object and to provide guaranteed performance levels for groups of processors, disk, memory, and fabric. Feldman told El Reg that last June that virtualized x64 servers cannot do this because they often oversubscribe resources to drive up utilization. Each core in the original blade runs at 1.6 GHz and has two threads and a SODIMM memory slot. Because the Z530 is a 32-bit processor, the system main memory was limited to 2 GB of operating system addressable memory for each server node on the board. The server card for the SM10000 is 5 by 11 inches.

Intel has not announced its latest dual-core, four-thread Atom N570 processor for netbook and notebook computers yet, but SeaMicro, a server maker, is getting its hands on the chip first. (The N570 was expected to be launched in February but apparently has been pushed out a few weeks.)

The dual-core chip runs at 1.66 GHz and, more importantly, has 64-bit memory addressing and support for the VT-x electronics that Intel created for its chips to help support virtualization hypervisors such as Xen, KVM, and Hyper-V. While the N570 could, in theory, support terabytes of main memory, the initial memory controller on the chip only supports 4 GB of addressable system memory. That’s twice the memory per chip as the Z530 and the two cores work, in essence, like a baby SMP server with about 1.8 or 1.9 times the oomph.

Xeon and Opteron code? No problem
The updated SM10000-64 server card is still 5 by 11 inches, and it has four of the dual-core N570 processors, so you can’t get twice as many cores in the box. You are still limited to 512. But what you really have are 256 server nodes that are basically twice as powerful with twice the memory scalability, and – here’s the kicker – they that can run 64-bit code compiled for Xeon or Opteron processors. Here’s what the Atom N570-based SeaMicro board looks like:

The new SeaMicro Atom N570 server board for the SM10000-64 server

Feldman realizes that there is plenty of empty real estate on the new server board and says that the company just wanted to get support for this 64-bit Atom out the door fast to capture the market opportunity. Supporting only 32-bit Linuxes is an issue for SeaMicro’s potential customers, since it forces a recompile. By the way, SeaMicro only supports Linux on the machines – just about any flavor you can name, include Red Hat Enterprise Linux and its CentOS clone, Debian and Ubuntu commercial distro were the ones that Feldman called out by name – but it don’t do Windows.

Both the old Z530 and N570 server boards plug into the chassis using absolutely standard PCI-Express 2.0 x16 slots, as you can see in the photos above.

From the outside, of course, the SM10000 and SM10000-64 machines look identical:

The SM10000 and SM10000-64 servers, front and side view.

The original SM10000 chassis had 128 PCI-Express 2.0 x16 slots, arranged in eight vertical columns, four on the left and four on the right of the chassis. You plug in 32 boards (two columns of 16) on each side to get your 512 Atoms per chassis. The new N570-based boards plug into the chassis in exactly the same manner. The old boards supported an aggregate of 1 TB of main memory across its nodes, and that has not changed with the new N570 nodes; the difference is that the new nodes have 4 GB of memory each and there are half as many in the box. (It is not clear if the two styles of nodes can be mixed and matched in the same chassis.)

The chassis has different network modules, which offer 8 to 64 Gigabit Ethernet uplinks or 2 to 16 10 Gigabit Ethernet uplinks per chassis. The FPGAs implementing the load balancer and terminal software as well as the switching software are in the chassis, not on the server cards. The chassis has room for 64 SATA or SSD disks as well, which are on the front of the box.

The base configuration of the original SM10000 came with 512 cores, 1 TB of memory, and a few disks; it was available at the end of July last year and cost $139,000. The new SM10000-64 uses the N570 processors, for a total of 256 chips but 512 cores, the same 1 TB of memory, eight 500 GB disks, and eight Gigabit Ethernet uplinks, for $148,000. Because there are half as many chipsets on the new box compared to the old one, it burns about 18 percent less power, too, when configured and doing real work.

Feldman is not going to give away his product roadmap, but he is very enthusiastic about the Atom family, particularly now that it runs 64-bit code, supports virtualization, and offers more main memory per socket. But don’t get the wrong idea. SeaMicro is watching all of the developments in the low-power chip racket.

“We built the system to support any CPU,” Feldman tells El Reg, and say that the N570 is the best chip currently available for the SeaMicro servers in terms of performance per watt, bar none. “This is Intel’s mantle to lose. They have to keep the crown.”

Not for SeaMicro’s sake, but for Intel’s own sake. The SM10000 design can easily adapt to future x64 chips from Advanced Micro Devices (can you say “Bobcat?”), as could any one of a number of variants of the ARM RISC processor popular in smartphones, tablets, and netbooks. While supporting ARM will require customers to do a recompile, with Apple, Google, and Microsoft standing behind ARM chips, it is not possible to ignore them. Especially if the ARM chips offer substantial savings in terms of money and power consumption.

Feldman is not going to tell us how many boxes he has shipped or how much money the initial systems have generated, of course. But he did confirm that companies such as Skype. France Telecom, China Netcom Broadband, and Rogers Communications, as well as Mozilla and Oak Ridge National Laboratories, had the SeaMicro machines in production. The company has 75 employees and is adding a new one every week, and it has various managed hosting, cloudy infrastructure, and telcos that want to remain anonymous using its boxes as well. Feldman said that “tens of millions of Internet users traverse SeaMicro equipment daily,” for whatever that is worth.

The real question is this: Has Google bought any of the new machines, and if not, why not? ®

This article originally appeared in The Register.