Sign up for our newsletter and get the latest HPC news and analysis.

Amazon EC2 Cluster Workload Management

We posted an article earlier today about the latest service offering from Amazon’s EC2 cloud resource.  Amazon has added what amounts to pseudo-tightly-coupled cluster platform support such that applications have some notion of quality of service [QOS].  MPI is generally not very friendly to link/node failures.  As such, this is directed specifically towards enterprise-level high performance computing applications.

That being said, how do you manage such a resource on the cloud?  Theoretically speaking, the cloud offerings loosely coupled compute “agents” in a distributed manner such that you really don’t care.  However, we, as HPC technologists, DO care.  The folks over at Clustercorp have created a series of integrated solutions such that one can now manage “clusters” of virtual nodes in the cloud the same way one manages a local set of physical nodes: Rocks+.

From their website:

Clustercorp’s Rocks+ and Amazon EC2 combine to form the ideal environment for running large sets of heterogeneous servers in the cloud for any number of general data center use-cases. Rocks+ provides users a ready-to-launch Rocks+ AMI, which is subsequently used to launch 10s, 100s, or 1,000s of servers with a single point of management and control.

The Rocks+ AMI comes pre-loaded with “Rolls,” which allow users to quickly build web servers, database servers, compute servers, and more, with pre-packaged software including Apache Web Server, MySQL, Java, and more, running on CentOS Linux.

Whoa!? Now you’re telling me the same management tools used in my local machines can be used in the cloud?  Yea buddy.  Management headaches are usually the problems that most integration teams easily overlook and often spend the most time mitigating at the end of the day.  Clustercorp’s ability to manage the local resources in the same manner of remote, virtual resources is incredibly powerful.

If you’re interested in utilizing the Clustercorp Rocks+ packages, check out their website here.


Trackbacks

  1. [...] Amazon EC2 cluster workload management [...]

Resource Links: