16 Reports Evaluate Clouds for Scientific Applications

Print Friendly, PDF & Email


Kate Keahey over at the Science Clouds blog has posted a collected set of 16 reports evaluating the performance and cost of scientific applications on Infrastructure as a Service (IaaS) clouds. The reports look various aspects of performance including low-level benchmarks, applications of various types, reliability, and cost.

My personal favorite is “Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud Amazon Web Services Cloud” — on top of the list since, having just come out at CloudCom 2010 last December it is the most recent. The authors evaluate the AWS IaaS offering based on the NERSC benchmarks framework — a comprehensive set of benchmarks capturing the typical workload in a scientific datacenter. They report not only the performance characteristics of scientific applications on virtual clusters created in the cloud but also note the mean time between failures (MTBF) of a virtual cluster deployed on cloud resources — the consequences of which I (coincidentally) blogged about around the time this paper was presented.

Keahey requests that readers aid in her ongoing research by sending in links to new papers and reports. Full Story


  1. […] This post was mentioned on Twitter by insideHPC and techilatech, NM Tech Council. NM Tech Council said: RT @insideHPC Just in: 16 Reports Evaluate Clouds for Scientific Applications http://bit.ly/dGee5c #HPC […]


  1. This is a great list and we’re thrilled that a Berkeley Lab paper was called out as favorite. Thank you.

    However, while this is indeed the most recent paper, it does not contain the most recent information. As we all know, the cogs of peer review grind far more slowly than companies such as Amazon can change their offerings.

    Amazon had improved its cloud services — and its benchmarks — by the time this paper was presented. In fairness, Keith Jackson, the author who presented the paper at CloudCom10, included some of the new results in his presentation, archived here: http://screencast.computer.org:8080/ess/echo/presentation/6cb2953a-3143-4775-b165-8de7af1ba88a

    So, while we’re thrilled that our researchers are getting the recognition that they deserve, we’d like to point out this information rather than appear to be unfair to anyone.