Feldman comments on HPC in the cloud

Print Friendly, PDF & Email

Michael Feldman brings together several examples of working HPC use in the cloud in a recent blog piece at HPCwire

For example, the company was able to rent CPU cycles on EC2 to run a bioinformatics sequencing code on a 64-node EC2 cluster. For a 20 minute run, the cost to Eli Lilly was $6.40. That’s hard to beat when compared to the price of maintaining those additional 64 compute nodes on a permanent basis.

For bioscience businesses, the cloud story is especially compelling. Unlike other traditional HPC users like government labs, financial services firms, and oil & gas companies, life sciences came relatively late to the information technology game, so computing know-how and infrastructure at these companies tend to be spread rather thinly (at least relative to, say, a DOE lab). But today biotech companies are fully immersed in and dependent upon information technology, especially high performance computing.

Michael also touches on a very real concern, satisfying the real security needs of companies and organizations with a mission to protect their data (that’s the real security needs — the security weenies will drum out all sorts of reasons “you can’t do that” that mostly boil down to “because we’ve never thought about it before;” these are real problems, but not real security problems)

The early experiences by these drug firms also point to how security concerns are holding back more widespread use of cloud computing. In this case, their main concern is protecting their intellectual property and patents, but almost all HPC users (not to mention just everyday enterprise users) have security issues of one sort or another. It’s worth noting here that Verizon’s new cloud platform offers added security, primary because their cloud runs over their own private network. But they also offer additional security in the form of identity and access management, host intrusion detection, application vulnerability assessment, network application assessment and professional security services. It’s not too hard to imagine that computing in the cloud can be made at least as secure as it is behind a local firewall.

Of course there are many who feel that HPC is solely about getting to each and every drop of performance, and running HPC applications in the cloud will never be as efficient as owning the iron, so it will not work out in the long run. Michael’s take matches my own

My take on this is that focusing on performance and computing efficiency ignores the more useful (but more slippery) concept of productivity. I’ve yet to see a research paper look at HPC in the cloud from this perspective.

While you certainly can’t argue with the benchmark results that demonstrate lower performance in some cases, I think the key point is to contrast some performance penalty for using clouds with the much greater penalty of not doing the computation at all. Also, there is a real cost to maintaining a datacenter that extends all the way from the administrators and the power bills to the headaches in getting a literal act of Congress (as required at many federal institutions) before doing significant upgrades to existing facilities or adding new footprint. I think that those that argue that cloud computing is on its face not useful are just as wrong as those that argue it is the universal panacea for all computing ills.