I’m particularly interested in the “System-level portability” usage of VMs in HPC that you mentioned. A great use of this concept is Globus’ Nimbus toolkit [1] and its usage to create Science Clouds [2]. This would lower the barrier to getting more applications into a cloud or on a computational grid.

Deploying an application’s required environment as a VM to remote clusters in a cloud or computational grid will save time both on the part of the application developer and user (fewer wrapper scripts and conditionals and job restarts due to mismatched environments) as well as the time needed from support personnel and system administrators at the local and remote sites (where support tickets are opened and servers are tweaked to accommodate new applications).

Perhaps if those savings can be quantified it can justify the purchase of additional hardware to offset any loss in performance that results from VM usage.

[1] http://workspace.globus.org/
[2] http://workspace.globus.org/clouds/

There are language solutions to these problems, but there are not only language solutions, and the best solution (from a usability perspective) is one that requires no change by the user.

I’m unclear how VMs are perceived as a solution here. There is research in this area, but it is perceived as a HPC language issue, i.e. what are the best abstractions we can provide to users to easily, effectively, and robustly express a wide range of problems on highly-parallel machines?

I’m all for increasing funding in this much neglected area (although DARPA HPCS did help a bit).

I’ll be careful about applying the VM koolaide as a solution to this critical issue though. I think the benefits of VMs are really in the data center management arena.

IMHO

Excellent comments. I totally agree. I guess we are all speed junkies and we need to get off the crack to make our lives better But it doesn’t help when government agencies that fund research and machines focus totally on performance and price as the measures of either funding the research proposal or selecting a vendor to build a machine. For example, I wish NSF would start funding open-source projects to solve some of these problems.

Keep the blogs and ideas coming – I like to see things shaken up

Thanks!

So, I’m being a little facetious (but only a little) when I say “FLOPS be damned.” Clearly, 30% performance degradation is a big deal. I guess my real point is that we have big usability issues in HPC that we continue to ignore while we strive after FLOPS. 20 years ago this may have been appropriate, but today I think the situation is analogous to continuing to tweak the engine in a car we’ve been driving for 20 years with no seat. Let’s put in a seat.

I realize that you aren’t arguing against my point, but rather observing that there are real problems to be overcome. I agree. I’m just afraid that no one is going to take the time to overcome them because they lay off the path to higher immediate performance.

Ironic that this is twice in a week where you have posted an article that has great relevance to the situation at my site.

Last week we had a Dell Rep telling us Infiniband was on it’s way out and we needed to seriously look at moving the data center to 10 Gig E in the future. (replacing all of our existing infrastructure of course).

And now the discussion on Virtualization. It’s a big topic here at our site and the management staff has drunk the VM koolade. Ignore the man behind the curtain, it will all work as advertised. It’s fast, it’s reliable, it’ll make you breakfast in the morning. It fits all situations in the data center. (-:

VM has it’s place in the data center surely, probably lots of places. But the hype being sold us is almost funny. It will be interesting to see what plays out. I’ll continue to read the various links and consolidate some information. Maybe I’ll have a better informed comment later as opposed to just an observational one.

Just thought it was funny that insideHPC nailed two hot topics for us in less than a week.

Cheers.

Rich

I like the concept of virtualization and I agree with most everything Oak Ridge said (haven’t read the trip report yet). Great ideas and it would make life easier on so many fronts, including the idea of moving or replicating VM’s on the fly. Laudable goals and I would love to see things move along.

But, currently, the performance when running on VM’s really isn’t great. I’m not talking about a couple of percent. The last tests I’ve seen indicate that you take a network performance hit on the order of 20-30% when running in a VM. That’s pretty severe.

Second – I know of one experiment where a single node was running a code doing some local IO and moved to another node. It was running a very popular cluster package along with Xen. It took about 30 minutes to move the application from one node to another. This case isn’t as strange as you would think. Writing to centralized storage is a good thing, but for some applications, it’s actually better to write to local storage.

Third – one thing that is done with HPC applications to get better performance and more predicatable performance is the idea of pinning the process to a core. But VMWare specifically says not to do this because of various problems in moving a process from one node to another. While VMWare isn’t all VM’s today they have a great deal of experience.

Like 10GigE, I would love to see these problems and others solved to allow virtualization in HPCC. There are just some problems that need to be overcome today.

BTW – here’s a blog I wrote a while ago about the concept.

http://www.delltechcenter.com/page/6-02-2008+-+Whither+Virtualization+in+HPCC+-+Comments

I don’t know if it helps or not