Sign up for our newsletter and get the latest HPC news and analysis.

Goethe University building a GPU cluster, but not how you think [UPDATED]

AMD’s John Fruehe posted on his blog yesterday that Goethe University is building a GPU-CPU hybrid cluster, but not in the usual (i.e., NVIDIA) way. They are using Mangy Cours processors with AMD’s GPU, which is a much less common choice

AMD logoThis cluster will combine one thousand five hundred and forty four (1,544) 12-core AMD Opteron processors (that’s a total of 18,528 cores) and 772 AMD GPUs. That amounts to one GPU card per 2P node.

To help bring the whole set of servers together into a cohesive cluster, Bright Cluster Manager(tm) software will be utilized and Mellanox quad data rate InfiniBand will serve as the interconnect between the nodes.  To help maximize data center space, the cluster will be built on a SuperMicro “twin” platforms that provide 2 motherboards in a 2U rack server chassis.  This allows the 772 total motherboards to be located in 386 physical servers.  To give you an idea of the size of this cluster, 386 2U servers will fit in about 18 42U racks if you were to put in only the servers (and no networking equipment.)

[John corrected his post, and we subsequently changed our post, following a set of comments from an insideHPC reader who noticed some errors leading to the correction. See the comment stream below for details.] The university is partnering with ClusterVision, who we’ve written about before.

Fruehe posted this under the headline “Fusion for Servers Happening Today,” which of course it isn’t. The point of the fusion project is to put the GPU on the same die as the CPU, and this is very much the standard add-on card approach. Not only is fusion for servers not happening today, AMD won’t even talk about when it might happen. But what can you expect from an AMD marketing guy?

Comments

  1. So let me get this right. 1U server with two dual socket motherboards and then a GPU for each of those nodes? All in a 1U? Not only does Supermicro not make a 1U Twin AMD platform, they don’t make a twin AMD platform with room for two GPU’s either. At least they have not publicly stated these platforms and as of right now, this info is not on their site.
    And then there is this comment:
    “386 1U servers will fit in almost five (5) twin 42U racks if you were to put in only the servers”
    What? This statement doesn’t even make sense. What is a twin 42U rack anyway? An 84U rack? 5 42U racks equals 210 available U’s, if you will. And no, 386 1U servers don’t fit in 210 U’s. Who posts this stuff anyway?

  2. John West says:

    swanny – your comments are on AMD’s blog post, so I’ve asked them to provide some clarifying details. I’ll post back when I hear from them.

  3. wow…two minutes of my own research and here is what I found. FIrst off, the cluster deal was won by Clustervision. Their own press release here:
    http://www.clustervision.com/pr_frankfurt_uk.php
    Here is what the press release says in part:
    “The supercomputer will be built by ClusterVision using Bright Cluster Manager™ as the cluster management software, which is unique due to its built-in functionality for large and complex HPC clusters. At the heart of the cluster will be 386 units of 2U Supermicro GPU-optimised Twin servers with two hot-plug nodes each, 1544 AMD Opteron 12-core “Magny-Cours” CPUs, 772 ATI Radeon GPGPUs, and QDR Mellanox InfiniBand. The supercomputer will also have 420 TeraBytes of disk storage from Dell with an extremely fast, 10GB/s, parallel FhGFS filesystem from the Fraunhofer Institute ITWM in Kaiserslautern.”
    Too bad the AMD guy can’t seem to cut and paste like I can…..:)
    geez……

  4. This is all OLD news anyway. The press release is date 1-June 2010.

  5. John West says:

    Update back from John Fruehe at AMD. Turns out that these are indeed 2U, not 1U servers as the post originally indicated, and that the rack count was wrong. John has updated his post, and the corrected text reads as follows:

    “To help bring the whole set of servers together into a cohesive cluster, Bright Cluster Manager(tm) software will be utilized and Mellanox quad data rate InfiniBand will serve as the interconnect between the nodes. To help maximize data center space, the cluster will be built on a SuperMicro “twin” platforms that provide 2 motherboards in a 2U rack server chassis. This allows the 772 total motherboards to be located in 386 physical servers. To give you an idea of the size of this cluster, 386 2U servers will fit in about 18 42U racks if you were to put in only the servers (and no networking equipment.)”

    I’ve also updated this blog post to reflect the change. AMD was very appreciative that our readers are so on the ball, as am I.

Resource Links: