The Yahoo! M45 project (also part of the Open Cirrus cloud effort funded by Yahoo!, HP, and Intel and hosted at facilities around the world) just took a pretty big step forward. Researchers at Carnegie Mellon have been using M45 since November of 2007; now four new universities are joining the effort
Yahoo! Inc., a leading global Internet company, today announced it has expanded its partnerships with top U.S. universities to advance cloud computing research. The University of California at Berkeley, Cornell University and the University of Massachusetts at Amherst will join Carnegie Mellon University in using Yahoo!’s cloud computing cluster to conduct large-scale systems software research and explore new applications that analyze Internet-scale data sets, ranging from voting records to online news sources.
The cluster has about 4,000 cores and 1.5 PB of disk, and runs Hadoop, “an open source distributed file system and parallel execution environment that enables its users to process massive amounts of data”
“We have been using the Yahoo! cluster for more than a year now and have made significant progress in a number of key research areas, resulting in the publication of more than two dozen academic papers,” said Randal E. Bryant, dean of the School of Computer Science at Carnegie Mellon. “Our researchers were able to extract and process documents from the Web in a way that was not possible before, changing the way we think about research problems. We were also able to conduct research over a corpus of 200 million Web pages, processing two orders of magnitude more data. We conducted systems software research, comparing, for example, the performance of the Hadoop file system and other parallel file systems. The simultaneous access to applications and systems software has been a real benefit and we look forward to our continued partnership with Yahoo! and joint contributions to the cloud computing community.”
How does this relate to Yahoo!’s other computing research efforts?
Partnership with these universities is the next step in expanding Yahoo!’s leadership in supporting cloud computing research. In July 2008, Yahoo! joined forces with HP, Intel, the University of Illinois at Urbana-Champaign, the Infocomm Development Authority (IDA) in Singapore, and the Karlsruhe Institute of Technology (KIT) in Germany to create Open Cirrus™, a global, multi-data center, open source testbed for advancing cloud computing research and education. The partnership with Illinois also includes the National Science Foundation, creating a cloud computing cluster that is made available to the entire reach of the NSF academic community. The international partnership promotes open collaboration among industry, academia and governments by removing the financial and logistical barriers to research in data-intensive, Internet-scale computing. As the Yahoo! M45 cluster is part of the Open Cirrus cloud computing testbed, the above universities will also gain access to and be part of the Open Cirrus community.