Intro to HPC: what’s a cluster?

A high performance computer appropriate for most small and medium-sized businesses today is built from what are basically many ordinary computers connected together with a network and centrally coordinated by some special software. Because the computers are usually physically very close together, the common term for a high performance computer today is a cluster, and when you call up your friendly IT geek you’ll be talking with them about helping you install a small cluster for your business.

Many different shapes and sizes

When people talk about the size of an HPC cluster they are usually referring to how many processors, or how many cores, it has. Of course you know what a processor is already, but you may not be used to the term cores.

From the beginning mainstream processors from companies like Intel had one “brain” in them. You may remember names like 386, 486, or Pentium — all of these processors had a bunch of circuits in them that all added up to make one processor, stuck in one socket on the motherboard in your computer.

Today modern processors, even the ones you find in your laptop and on your desk, don’t just have one “brain” in them. As computer circuits have gotten smaller designers have figured out how to cram 2, 4, 6, or even 8 “brains” onto a single chip that plugs into one socket on your motherboard. It’s as if you had many processors all squeezed into one chip. In slightly more technical (and more accurate) terms, these “brains” are called cores, and multicore chips today are commonly quadcore (if they have 4 cores, as does Intel’s recent generation of Xeon chips), and six-core (as in AMD’s recent generation of Opteron chips). The number of cores that designers can fit onto a single chip is doubling about every 18 months, so these numbers will change fairly frequently.

How do you know if you need a small, medium, or large cluster? That depends upon your applications and the job you expect your cluster to do. Read more about that in What can HPC do for my business? or The big picture: where does HPC fit?.

So, what sizes are most likely to be of interest for your business?

Personal-sized clusters

One of the most convenient shapes for small clusters today is the deskside chassis. Deskside clusters come in a chassis that you can plug into the wall on your office, and they are designed to sit on the floor next to your desk. The chassis can hold a relatively small number of computers that are on blades, trays, or in enclosures that slide into the chassis and bundle everything together.

Note that I said the number of computers is relatively small. This is relative to the maximum size of clusters today, which can be in the tens of thousands of processors — almost certainly much larger than anything you’d want for your business.

Deskside clusters from companies like HP, SGI, Cray, and others can hold up to a few hundred cores, a size that is likely to be entirely adequate for most small business needs.

Stepping up

Many companies that offer deskside clusters also offer the ability to connect two or more of them together to make a larger, mid-sized cluster. In the case of Cray’s CX-1, for example, you actually stack several of the deskside chassis on top of one another and connect them together with additional network connections to make them act like one larger cluster (see the image). In other cases companies offer larger chassis that hold more computers, including mid-height racks that would still be appropriate for an office environment, and full-sized (42U) racks that are standard issue in computer rooms around the world. In some cases you can get nearly 800 cores in racks like these and still have a solution that may be appropriate in some office environments (though it will start to get a little noisy, and you may have to have your electrician run dedicated power circuits).

Going all in

If your computing needs are fairly large — say, for example, if you need to run many highly complex engineering simulations on large models — then you may need a cluster with 1,000 or more cores. At this point you will want to set aside some dedicated space for your cluster in a space that’s configured to meet the special cooling, power, and maintenance needs of large IT resources. If your needs are this large you probably already have an IT group that can help, and there are many integration contractors and even vendors that are eager to help you plan a larger cluster installation. It can actually be a little complicated to do this right, so do your planning ahead of time, and talk to as many people as you can find who are willing to help.


To learn more about successful HPC clusters download the insideHPC Guide to Successful HPC Clusters or read the article series the Five Essential Strategies for Successful HPC Clusters.