Preparing for Exascale: ALCF’s Aurora Early Science Program and Visualizing Cancer’s Spread

Print Friendly, PDF & Email

Image: Joseph Insley, Argonne National Laboratory

Scientists are preparing a cancer modeling study to run on Argonne’s upcoming Aurora supercomputer before it goes online in 2022.

The U.S. Department of Energy’s (DOE) Argonne National Laboratory will be home to one of the nation’s first exascale supercomputers – Aurora is scheduled to arrive in 2022. To prepare codes for the architecture and scale of the system, 15 research teams are taking part in the Aurora Early Science Program through the Argonne Leadership Computing Facility (ALCF), a DOE Office of Science User Facility. With access to pre-production time on the supercomputer, these researchers will be among the first in the world to use an exascale machine for science.

Over 60,000 miles of blood vessels carry nutrients throughout our bodies. At times, however, they can end up transporting something much more dangerous — cancer.

Cancer cells can spread from one part of the body to another by traveling through our blood vessels. Understanding how metastasis happens can guide development of new drugs and treatments. Randles’ team will use Aurora to create high-resolution representations of the geometry of the circulatory system and simulate the flow of fluid, as well as collections of red blood cells and tumor cells moving in the fluid flow.

Amanda Randles, an assistant professor at Duke University, is using modeling and simulation to find answers. She is one of a few researchers chosen to take part in the ALCF’s Aurora Early Science Program (ESP). Randles and her team have developed HARVEY, a model that simulates blood flow throughout the body, and now they want to use it to predict the movement of cancer cells at a microscopic level.

“Tackling our new research into the process of metastasis and performing the intricate simulations needed means we need even greater computing power to handle the massive data sets in real time,” she said. “The Aurora system will help us meet this need.”

Randles’ team is working with Argonne scientists to prepare their application to run on the new system. A major challenge: the disparity between computation speed and the speed at which data can be saved. Aurora will enable computation to happen at lightning speeds, but the rate at which data can be stored longer term (by being written to disk) can’t match this speed, a well-known gap that has only grown over the years.

Left unaddressed, researchers would have to slow down their computation by orders of magnitude to write all their simulation data to disk. But this approach would take much time and require massive storage. For Randles’ team and many others, this is unfeasible given the sheer volume of the data and workload.

As an alternative, the HPC community is engaged in an effort to enable data analysis to happen in situ (meaning in the computer memory) as simulations are running. Visualizing and analyzing data in this way enables computation to still happen quickly, while giving researchers the chance to decide what data should be stored long term versus thrown away.

“If, for example, you identify a particular area of interest within the whole system, you might choose to save data at a higher frequency or higher level of detail just in that small area. That would still enable you to get more science out of the data while reducing the amount of data that you actually have to write to disk,” said Joseph Insley, ALCF visualization and data analysis team lead.

Through a DOE project called SENSEI, Insley and ALCF computer scientist Silvio Rizzi are working with other researchers across the DOE laboratory complex and industry to create a unified interface through which scientists can access a library of frameworks that support in situ data analysis and visualization. With access to early hardware and the Aurora software development kit, the ESP team has been testing and developing these capabilities in advance of the exascale machine’s arrival.

“We’ve been using the ALCF’s Theta supercomputer and early Aurora hardware to integrate this library with the HARVEY code. By enabling visualization and analysis on the data while it’s still in memory, the science team will be able to gain more insights from the data than they otherwise would,” said Insley.

Ultimately, the team’s research on Aurora aims to inform the development of new cancer drugs and treatments that have the potential to save lives in years to come.

“By understanding the biological mechanisms behind metastasized cancer cells, too, we hope our work with HARVEY will eventually help doctors and their patients in the fight against cancer,” Randles concluded.

Randles’ ESP project, ​“Extreme-Scale In-Situ Visualization and Analysis of Fluid-Structure-Interaction Simulations,” is supported by DOE’s Advanced Scientific Computing Research program.

source: U.S. Department of Energy Science News Source