Kate Keahey over at the Science Clouds blog has posted a collected set of 16 reports evaluating the performance and cost of scientific applications on Infrastructure as a Service (IaaS) clouds. The reports look various aspects of performance including low-level benchmarks, applications of various types, reliability, and cost.
My personal favorite is “Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud Amazon Web Services Cloud” — on top of the list since, having just come out at CloudCom 2010 last December it is the most recent. The authors evaluate the AWS IaaS offering based on the NERSC benchmarks framework — a comprehensive set of benchmarks capturing the typical workload in a scientific datacenter. They report not only the performance characteristics of scientific applications on virtual clusters created in the cloud but also note the mean time between failures (MTBF) of a virtual cluster deployed on cloud resources — the consequences of which I (coincidentally) blogged about around the time this paper was presented.
Keahey requests that readers aid in her ongoing research by sending in links to new papers and reports. Full Story