By: Dr. Scott Yockel, University Research Computing Officer at the Harvard University and Scott Tease, Vice President and General Manager of HPC and AI at Lenovo
Cannon’s mission is clear – to be the computational powerhouse behind some of the most groundbreaking research of this world and beyond – from the impact of environmental pollutants on human health to the intricate study of black holes. Developed for the world renown researchers at the Harvard University, the Cannon HPC system and the Faculty of Arts and Sciences Research Computing (FASRC) group supports over 800 labs spanning disciplines such as science, engineering, and public health by enabling research from genome sequencing to climate justice.
Expanding the Breadth of Research
Social science and public health research in the days before Cannon were slow-going, using minimal compute power of individual laptops and workstations. But with the birth of Cannon, the realms of capacity, speed and performance paved the way for new discoveries.
Economics researchers use Cannon to work on an initiative called Opportunity Insights whose mission is to identify barriers to economic opportunity and develop scalable solutions that will empower people throughout the United States to rise out of poverty and achieve better life outcomes. To support the development of these solutions, Cannon houses economic, census and many other reputable data sources in the cluster’s Security Enhanced Environment. Scientific research is conducted using “big data” to improve upward mobility and work collaboratively with local community stakeholders to translate the research findings into actionable policy change.
Within the Harvard T.H. Chan School of Public Health, the National Studies on Air Pollution and Health faculty and researchers compile and analyze census, healthcare and pollution data to study how concentrations of air particles contribute to respiratory illness and disease over time. The insights revealed from this analysis is regularly shared with the Environmental Protection Agency (EPA) to enact stronger and stringent environmental safety policies. Additionally, Black Hole Initiative researchers use parallelization to refine telescope images from the deepest reaches of space. They start by running their workload simulations on Cannon first to hone and perfect their models before moving on to an even larger supercomputer, such as Leibniz Supercomputer Center LRZ in Munich, to run the simulations at a larger scale.
The Power Behind the Curtain
Cannon is a heterogenous HPC system comprised of 80,000 cores. The last two generations deployed included Lenovo ThinkSystem with 864 SD650 standard compute systems in summer of 2019, and a recent expansion of 98 SD650 V2 “bigmem” compute nodes and 46 SD650-N V2 GPU nodes. All the computational power of an HPC system of this magnitude is bound to emit a significant amount of heat. Cue Lenovo NeptuneTM liquid cooling technology. Dr. Scott Yockel, University Research Computing Officer at the Harvard University, said it best, “Going the liquid cooling route is making a choice for the future.” According to Scott, before the launch of Cannon, the processing power of previous generations of HPC clusters had hovered around 2GHz performance with a decent number of cores for many years without substantial improvements. But, with Lenovo’s liquid cooling technology, Cannon can operate at a higher frequency, (up to 3.4GHz), and at a higher core count while maintaining thermal stability. “It was important for Harvard to provide both high core count and high processor frequency for the diverse researchers we serve,” Dr. Yockel said.
Harvard FASRC was forward thinking, with the desire to go beyond the limits of current traditional air cooling. As future generations of accelerators emerge, the ability to use air cooling at the required performance levels and rack density will be essentially eliminated. So, instead of being forced down the path of liquid cooling in the future, Dr. Yockel had the foresight to get ahead of the game and enable it now. Liquid cooling technology like Lenovo’s NeptuneTM will allow the researchers the ability to reap the benefits of powering a full rack of GPU nodes (36 four- GPU servers).
But the benefits of liquid cooling technology don’t end there. An interesting by-product of standing up dense, liquid cooled servers such as the Lenovo ThinkSystem SD650-N V2, was the ability to increase footprint in the data center space. By eliminating air chillers and hot isle containment, the data center in the FASRC can now fit up to 16 racks per row, a 33% increase! “If we can change to this amount of density and cool it, we don’t have to build another building,” Dr. Yockel quipped. Additionally, plumbing and electrical supply can support even more racks resulting in reduced overhead from facility costs.
The future of Cannon is bright as it is uniquely designed with extra chassis space for expected growth. One initiative on the horizon is conducting research in Harvard’s new Kempner Institute for the Study of Natural and Artificial Intelligence. According to The Harvard Gazette, this University-wide initiative stands at the intersection of neuroscience and artificial intelligence. The research will result in an enhanced understanding of how humans think, perceive the world around them, make decisions, and learn, thereby advancing the rapidly evolving field of AI.
Cannon is poised and ready to support the computational needs of the new Kemper Institute, current and future research of Harvard at the Massachusetts Green High Performance Computing Center and any other new, innovative discoveries that require its vast compute power.
