In this Sponsored Post, our friends over at Altair explain that to keep the world’s weather sites running smoothly, Altair and the Cylc open source community have packaged an industry-leading workload manager, Altair PBS Professional™, together with the Cylc workflow engine plus other helpful plug-ins to create the Altair Weather Solution.
Weather impacts everyone on the planet. Knowing what to expect in advance — whether it’s sunny days, fog, thunderstorms, tornadoes, or the destructive force of a hurricane — enables us to make informed decisions and save lives. Long-term prediction of our changing climate allows us to extrapolate based on past and current environmental conditions, to examine interactions between atmospheric and terrestrial variables, and to understand how this complex web of factors is likely to impact life on Earth.
High-performance computing (HPC) is a critical enabler for weather forecasting and climate modeling. Major weather and climate centers around the world use HPC technology to power numerical weather prediction (NWP), climate prediction, ocean modeling, and more. As HPC technology gets smarter and faster, model fidelity increases and researchers can run more simulations in less time, giving accuracy a big boost.
Modeling and predicting our world’s weather and climate requires super-powered HPC hardware, but it’s not all about the big iron. Effective, targeted software is required to efficiently run the complex, demanding, and high-stakes workloads that help us stay a step ahead of Mother Nature. To keep the world’s weather sites running smoothly, Altair and the Cylc open source community have packaged an industry-leading workload manager, Altair PBS Professional™, together with the Cylc workflow engine plus other helpful plug-ins to create the Altair Weather Solution.
Orchestrating Cycling Workflows with Cylc
Cylc is an open-source Python workflow engine for cycling systems that handles a range of workflow complexities. It automatically executes tasks according to detailed schedules and dependencies and is especially useful in areas such as weather and climate modeling, NWP, physics simulation, and data processing. Cylc has become a popular choice at major weather and climate centers around the world.
With Cylc, tasks can be run as soon as dependencies are met, with maximum concurrency for more efficient use of HPC resources. Cylc was originally developed for operational environmental forecasting at the National Institute of Water and Atmospheric Research (NIWA) in New Zealand and is now an open-source collaboration between NIWA, the Met Office (UK), the Center of Excellence for Weather and Climate Simulation in Europe (ESiWACE), Australia’s Bureau of Meteorology, and other contributors.
Cycling is not the same as real-time scheduling,” says NIWA’s Dr. Hilary Oliver, the originator of Cylc. “Cylc seamlessly transitions between scheduling as quickly as possible when running off the clock and real-time scheduling once the system has caught up to the clock.”
Altair’s Weather Solution: Cylc and PBS Professional
The Bureau of Meteorology runs its NWP suite on its petascale Cray supercomputer. When the Bureau needed a modern, scalable service to support its growing NWP modeling and post-processing output, it partnered with NIWA and Altair to develop an environment for monitoring the performance of many Cylc workflows, including workload orchestration by PBS Professional, part of the Altair PBS Works™ suite. It allows HPC users to simplify HPC infrastructure management and optimize system utilization, improving application performance and maximizing ROI on hardware and software investments.
David Block, project manager at Altair, says, “Altair has a long history of listening to our customers and producing robust solutions on time and on target. Altair is collaborating with the Bureau to build a general-purpose solution to assist weather and climate staff with monitoring their environment, and then providing the diagnostics necessary to pinpoint the most complex issues.”
Altair was selected to assist the Bureau because of its knowledge of HPC systems and our need to integrate with their products,” says Bureau program director Tim Pugh. “PBS Professional provides the job resource management and scheduling on our HPC system. It’s critical to the system such that if that service were to go down, then we’d consider that a whole-system outage. Altair understands the criticality of that.” The partnership, he said, “allows us to achieve the outcomes every day that are needed by the Bureau’s forecast center.”
Altair also provides commercial Cylc support.
Learn more about the Altair weather solution.
