A new EU research project called “ExaHyPE” is developing open-source software for exascale-class supercomputers. As an international project coordinated at TUM in Munich, ExaHyPE (“An Exascale Hyperbolic PDE Engine”) seeks to develop novel software, initially for simulations in geophysics and astrophysics.
Computer-based simulations drive progress in the field of science. In addition to theory and experiments, simulations have long since been crucial for acquiring knowledge and insight. Supercomputers allow for the computation of increasingly complex and precise models. The EU ExaHyPE project has an interdisciplinary team of researchers from seven institutions in Germany, Italy, the United Kingdom, and Russia, and integrates well into Europe’s strategy for developing an exascale-class supercomputer by 2020. In order to be able to leverage the incredible processing power of exascale systems for correspondingly comprehensive simulation tasks, the entire supercomputing infrastructure, including the software, must be prepared for such systems.
Supercomputing of the future poses immense challenges for the ExaHyPE researchers. Currently, the biggest obstacle for achieving exascale computing is energy consumption. Today, the world’s fastest supercomputers – Tianhe-2 (China), Titan (US), Sequoia (US) and the K Computer (Japan) – operate in the petaflop/s range (1015 computer operations per second) and require between 8 and 18 megawatts (source: www.top500.org), with the energy costs amounting to about US$ 1 million per megawatt and year.
“Based on current technologies, an exascale computer with a demand of close to 70 megawatts would represent both a financial and an infrastructural challenge,” explains ExaHyPE coordinator Professor Michael Bader of TUM. “That is why simulation software developed as part of the ExaHyPE project will be consistently designed for the requirements of future energy-efficient hardware.”
On the hardware side, extreme parallelization is to be expected. “By 2020 supercomputers will encompass hundreds of millions processor cores,” Bader adds. “At the same time, the hardware – which is pushed to its physical limits to achieve the further increase in performance and still must run as energy efficiently as possible – will increasingly tend to be plagued with interruptions and fluctuating performance curves. ExaHyPE will consequently examine the dynamic distribution of computer operations to processor cores – even if these fail while performing calculations.”
Another objective is to reduce the internal-hardware communication simultaneously with the parallelization. Each data transfer is implemented at the expense of energy consumption. In ten years, supercomputers will be able to run calculations 1000 times faster than today. However, memory access time will fail to evolve at the same rate. The used algorithms should be inherently memory-efficient and require as little data transfer as possible to ensure fast, energy-efficient computer operations.
In order to take full advantage of the smallest possible amount of memory, the consortium is developing new scalable algorithms, which dynamically increase the resolution of simulations, i.e. the implemented numerical observation points, wherever the computer simulation needs – and only there. As a result, scientists will be able to limit the necessary computer operations to a minimum while simultaneously achieving the greatest possible accuracy for the simulation.
Two application scenarios: Earthquakes and gamma ray explosions
The ExaHyPE researchers will prepare the new algorithms based on two application scenarios taken from geophysics (earthquakes) and astrophysics (gamma ray explosions). Earthquakes cannot be predicted. However, simulations carried out on exascale supercomputers could help us to better assess the risk of aftershocks. Regional earthquake simulations promise to provide a better understanding of what takes place during large-scale earthquakes and their aftershocks. In the field of astrophysics, ExaHyPE systems will simulate orbiting neutron stars which are merging. Such systems are not only suspected of being the greatest source of gravitational waves but could also be the cause of ”gamma ray explosions”. Exascale simulations should allow us to study these long-standing mysteries of astrophysics and see them in a new light.
In spite of the two precisely defined areas of application, the researchers want to keep the new algorithms as general as possible so that they may also be used in other disciplines after making corresponding adaptations. Examples could include the simulation of climate and weather phenomena, the complex flow and combustion processes in engineering sciences, or even the forecasting of natural catastrophes like tsunamis or floods. “Our objective is to ensure that medium-size, interdisciplinary research teams are able to adapt the simulation software for their specific purposes within a year of its release,” Bader says. To guarantee a rapid dissemination of the new technology, the consortium will release it as open source software.
Comprehensive expertise through international, interdisciplinary cooperation
The ExaHyPE project objectives call for an intensive cooperation of experts across many disciplines and country borders. On the German side, the consortium includes the Technische Universität München (Prof. Dr. Michael Bader, Informatics Department, High Performance Computing), the Frankfurt Institute for Advanced Studies (Prof. Dr. Luciano Rezzolla, Institute for Theoretical Physics, Goethe Universität Frankfurt), the Ludwig-Maximilians-Universität München (Dr. Alice-Agnes Gabriel and Prof. Dr. Heiner Igel, Department of Earth and Environmental Sciences), and the Bavarian Research Alliance (Dipl.-Ing. Robert Iberl, Unit for Information & Communication Technologies). Italy is represented by Università degli Studi di Trento (Prof. Dr. Michael Dumbser, Dipartimento di Ingegneria Civile Ambientale e Meccanica) and the United Kingdom by Durham University (Dr. Tobias Weinzierl, School of Engineering and Computing Sciences). The consortium is supplemented by the Russian supercomputer vendor ZAO RSC Technologies (Alexander Moskovsky, CEO).