Oak Ridge National Laboratory announced today that a new version of the Energy Exascale Earth System Model, or E3SM, is two times faster than an earlier version released in 2018.
Earth system models have weather-scale resolution and use advanced computers to simulate aspects of earth’s variability and anticipate decadal changes that will critically impact the U.S. energy sector in coming years. Scientists at the Department of Energy’s Oak Ridge National Laboratory are part of the team that developed version 2 of the model — E3SMv2 — which was released to the scientific community in September 2021.
“E3SMv2 delivered twice the performance over E3SMv1 when using identical computational resources,” said ORNL computational scientist Sarat Sreepathi, who co-leads the E3SM Performance Group. “This is a significant achievement as the performance boost is reflected while running the fully integrated earth system model and not just confined to smaller model components.”
The earth, with its myriad interactions of atmosphere, oceans, land and ice components, presents an extraordinarily complex system for investigation. Earth system simulation involves solving approximations of physical, chemical and biological governing equations on spatial grids at resolutions that are as fine in scale as computing resources will allow.
“Even with the addition of new features in E3SMv2 to the atmosphere model and how it represents precipitation and clouds, we still doubled the model throughput,” Sreepathi said. “To put it another way, we cut the computational run time or time-to-solution in half.”
E3SM simulates earth system variability and projects decadal changes impacting the U.S. energy sector. credit: E3SM, DOE
“E3SMv2 allows us to more realistically simulate the present, which gives us more confidence to simulate the future,” said David Bader, Lawrence Livermore National Laboratory scientist and lead of the E3SM project. “The increase in computing power allows us to add more detail to processes and interactions that results in more accurate and useful simulations than the previous version.”
Achieving these improvements required collaboration across the national laboratory system. Sreepathi, along with ORNL’s Gaurab KC and Youngsung Kim, accelerated the effort by creating a comprehensive monitoring framework called PACE, or Performance Analytics for Computational Experiments. The PACE web portal provided both an automatic data collection system and a streamlined interface for scientists to evaluate the performance of E3SM experiments executed on DOE supercomputers. These data facilitated feedback-driven E3SMv2 model development and allowed researchers to optimize their experiments.
“Using the PACE web portal helped the multi-laboratory team understand how new model features were impacting computational performance,” said Sreepathi. “We were able to accurately track the evolution of the model’s performance.”
The E3SM project reliably simulates aspects of earth system variability, including regional air and water temperatures, which can strain energy grids; water availability, which affects power plant operations; extreme water-cycle events, such as floods and droughts, which impact infrastructure and bioenergy resources; and sea-level rise and coastal flooding, which threaten coastal infrastructure.
In addition, the resolution has been refined due to more powerful computers. There are now two fully coupled configurations: 100-kilometer, or km, globally uniform resolution atmosphere model and a regionally refined model, or RRM, with a resolution with 25 km over North America and 100 km elsewhere. The refined mesh configuration is particularly well suited for DOE applications.
“Thanks to the performance improvements, the RRM configuration of E3SMv2 runs as fast as E3SMv1 did in its standard resolution configuration (100 km) a few years ago. We are essentially getting the much higher resolution for ‘free,’” said LLNL atmospheric scientist Chris Golaz.
The team is now conducting the simulation campaign with E3SMv2. Team members have already simulated several thousand years, and are planning to run several thousand more.
The project includes more than 100 scientists and software engineers at multiple DOE laboratories as well as several universities; the DOE laboratories include Argonne, Brookhaven, Lawrence Livermore, Lawrence Berkeley, Los Alamos, Oak Ridge, Pacific Northwest and Sandia national laboratories. In recognition of unifying the DOE earth system modeling community to perform high-resolution coupled simulations, the E3SM executive committee was awarded the Secretary of Energy’s Achievement Award in 2015.
In addition, the E3SM project benefits from DOE programmatic collaborations, including the Exascale Computing Project and research efforts in Scientific Discovery Through Advanced Computing, Climate Model Development and Validation, Atmospheric Radiation Measurement, Program for Climate Model Diagnosis and Intercomparison, International Land Model Benchmarking Project, Community Earth System Model and Next Generation Ecosystem Experiments for the Arctic and the Tropics.
The E3SM project is supported by the Biological and Environmental Research program in DOE’s Office of Science.
UT-Battelle manages ORNL for the Department of Energy’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. The Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit energy.gov/science.
