In this special guest feature, Josh Simons from VMware gives us his thoughts on the first annual ISC Cloud ’10 conference, which kicked off today in Frankfurt, Germany.
ISC Cloud ’10 opened today in Frankfurt with the promise of “delivering qualified and unbiased insights into the latest cloud trends, innovations, and service strategies for HPC.” The 200 attendees represent 20 countries with an even mix of attendees from research and industrial/commercial organizations.
Wolfgang Gentzsch, ISC Cloud General Chair, made the point in his welcome presentation that while many may view cloud computing as a revolutionary new approach to IT, the HPC community justifiably sees it as an evolution that started in HPC with clusters, distributed computing, and then grid computing.
Dan Reed, VP for Extreme Computing at Microsoft, in his wide-ranging keynote described the divergence that has occurred over the last several decades between HPC and enterprise IT. In his view, the compute ecosystem has moved away from HPC’s sweet spot due to commodity-driven development which in turn has been driven by the massive explosion of computing use from early eras of paucity to the current plethora of compute used so widely by consumers and and users with commodity compute requirements.
This divergence is also exemplified by the different programming paradigms adopted by each community, with HPC firmly entrenched in the C/FORTRAN/MPI camp (relatively static since the 1980s) versus the more dynamic growth of new programming methods within the enterprise — Ruby, Python, and numerous other frameworks. He also pointed out that enterprise application and framework developers have paid much more attention to application resiliency than has been paid within HPC where MPI with its brittle failure semantics is still the primary programming model for distributed applications.
He believes that there are many points of commonality between enterprise and HPC requirements, though he did feel that HPC interconnect requirements were significantly different from that of the enterprise or cloud. He also called out virtualization as a specific challenge issue for HPC application performance.
One theme that arose repeatedly throughout the day was that interconnect capability was seen as a primary differentiator between a generic cloud infrastructure for enterprise and one for HPC. I touched on this point briefly in my talk due to time constraints, but wish I had dwelled longer on this issue because I don’t agree with it. While it is true that current cloud deployments like EC2 offer (with the exception of their HPC islands) only 1 GbE, I would argue that there are some very compelling needs that will push virtualized environments (private first and then public) to embrace higher bandwidth and lower latency solutions. These needs are not HPC specific. Instead, the arise from two other areas. The first is an emerging need for this type of interconnect within the virtualization platform itself to accelerate live migration operations, storage operations, and transfer of state traffic for maintaining coherency between fault-tolerant VM pairs. The second, equally important, is to address the needs of an increasingly important class of distributed frameworks gaining popularity in the enterprise, among them Hadoop, GemFire data fabric, Memcached, Oracle Coherence, etc. It is these requirements that will ultimately drive adoption of more capable interconnects within virtual environments, including eventually cloud environments.
Style points to Arend Dittmer from Penguin Computing for introducing some humor into the event when he started his slide presentation with a very large (and scary) photo of Ozzie Osbourne, famed lead singer for Black Sabbath. The tie-in to Penguin? Ozzie recently agreed to have his DNA sequenced on the theory that he may have some genetic protections that helped him live through the outrageous excesses of his younger days and the computation was done on the Penguin on Demand infrastructure. You can find more information about Ozzie’s sequencing at Discovery News.
Randy Bias, CEO of Cloudscaling, was not shy about sharing his opinions (I wanted to say “biases”, of course) about cloud computing and made some excellent points. In his view, the difference between standard IT approaches and cloud IT is the same difference between a traditional assembly-line approach to building cars versus a robotic factory assembly line — it is primarily about massive automation to achieve high scale. He had some pithy lines that are worth quoting and remembering: “Silos kill scale.”, “Complexity is the enemy of scalability.”