TACC’s Frontera Supports Investigation of Subatomic Protons – ‘the Origin of the Mass of Objects’

A team of researchers are using the Frontera supercomputer at the Texas Advanced Computing Center (TACC) to crack open the proton, a fundamental building block of the atomic nucleus that is used, among other ways, as a medical probe in magnetic resonance imaging.

Frontera, the world’s fifth-ranked HPC system on the Top500 list and the most powerful university supercomputer, is supporting about 240 physicists in 12 countries and 24 institutions collaborating on the COMPASS experiment — Common Muon and Proton Apparatus for Structure and Spectroscopy — at particle physics laboratory CERN, the European Organization for Nuclear Research. The scientists are exploring the proton structure by breaking it apart in particle collisions using particle beams of the CERN North Area’s Super Proton Synchrotron and a spin-polarized fixed target.

The smashed insides of the proton require large detectors, which record and digitize information about the particle and store it in a special data format. Complex algorithms are then used to interpret the data.

“The spatial pattern and the velocities of the fragmenting particles allow us to create a dynamic picture of the proton and other objects composed of quarks,” said Caroline Riedl, a research assistant professor of nuclear physics at the University of Illinois at Urbana-Champaign (UIUC). With her UIUC group, Riedl is involved in the COMPASS polarized Drell-Yan program and was the COMPASS technical coordinator for the 2018 run.

Her team, which had used the Blue Waters supercomputer at the National Center for Supercomputing Applications to process petabytes of COMPASS data, has now extended  the research onto Frontera to boost analysis of the existing COMPASS data taken between 2015 and 2018. Riedl’s team together with the collaborating COMPASS colleagues have confirmed for the first time the theoretically expected sign change of the Sivers function in Drell-Yan scattering as compared to Deep-Inelastic Scattering.

This so-called Sivers TMD (“Transverse-Momentum Dependent”) distribution arises from correlations between proton spin and quark transverse momentum – quarks being subatomic particles bound together by gluons – and thus appears connected to quark orbital motion inside the proton. Observation of the sign change of the Sivers TMD is one of few NSAC (Nuclear Science Advisory Committee) performance milestones for DOE– and NSF-funded research in nuclear physics.

Workflow of COMPASS experimental data production.

The COMPASS experiment shoots a beam of pions (particles made of quarks) onto a fixed target. The aftermath is chronicled by 240 tracking planes that follow the path of the subatomic particles released. Here’s where the computational challenges get heavy.

“The procedure of finding particle tracks emerging from the interaction point and traversing hundreds of COMPASS detector layers is CPU-intensive,” Riedl said. The tracking procedure is one of the first steps in the data analysis. An additional highly CPU-expensive task is the sampling of about 2 percent of the data to determine efficiencies for the detector planes, according to Riedl.

Providing the data in a timely manner for physics analysis presents a hurdle.

“The challenge consists of parallelizing the submissions of the tracking code on the computing grid while respecting the system in terms of I/O and numbers of requested computing nodes. A typical production campaign requires about 50,000, ideally parallel, submissions of the tracking code,” Riedl said.

Caroline Reidl, University of Illinois at Urbana-Champaign

All in all, about 3 petabytes of COMPASS data has been moved from Blue Waters to TACC’s Ranch storage management system, which enables it to be analyzed on Frontera.

In addition to analyzing past COMPASS data, her team is using Frontera to design new detectors for the future COMPASS++ /AMBER experiment. This new facility at the M2 beam line of the CERN Super Proton Synchrotron will enable a great variety of measurements to address fundamental issues of quantum chromodynamics.

The proposed program covers measurements of the proton charge radius using beams of muons, elementary particles similar to the electron but with much greater mass; the spectroscopy of mesons and baryons by using dedicated meson beams; the study of meson and baryon structure via the Drell-Yan process; and eventually the fundamental quest on the emergence of hadronic mass.

Riedl is driven by fundamental questions at the heart of the proton. How do the quarks move inside the proton, and what is their orbital motion? How are quarks distributed in the proton? And how do quarks and gluons generate the large observed nuclear masses?

The latter question will be accessed by the future COMPASS++/AMBER experiment at CERN, according to Riedl.

“We run mass productions of COMPASS data on Frontera, determine detector efficiencies, and simulate COMPASS and COMPASS++/AMBER data. The simulated data play a central role in understanding subtle detector effects and complement the experimental data,” Riedl said. “Frontera will allow us to analyze the COMPASS data in a timely manner and at the precision required to obtain an absolute normalization of the data with the smallest possible uncertainties.”

Riedl hopes that improved analysis on Frontera will allow researchers to reach discoveries inside the proton faster than ever.

“Only Frontera will allow for the detailed simulations necessary to optimize instrumentation upgrades for the future COMPASS++/AMBER experiment,” she added. “Frontera is a leading-edge supercomputing system funded by the National Science Foundation that will allow U.S researchers to compete with international research teams.”

Riedl’s research fits into the bigger picture of understanding nuclear physics and quantum chromodynamics, the field theory of the strong nuclear force. She investigates questions such as how quarks and gluons form the nuclei of matter, and how protons can be described in terms of Parton Distribution Functions, “partons” referring more generally to quarks and gluons.

“The special feature of our experiments lies in the usage of spin-polarized particle beams on spin-polarized fixed targets,” Riedl said. “By introducing transverse quark momenta, spin, and orbital angular momenta into the formalism, proton substructure becomes similarly rich as the substructure of the hydrogen atom, which was first described in the 1930’s,” she added. “During the early decades of the 21st century, proton hyperfine structure has moved into the focus of spin physicists.”

Curiosity drives her work.

“Humans are and always have been curious to find out what holds the world together at its core,” Riedl said. “We attempt to unravel the origin of the mass of objects in our daily life and to map the dynamic quark structure of the proton.”

This research is supported by the NSF Office of Advanced Cyberinfrastructure Award # 1713684 : Mapping Proton Quark Structure using Petabytes of COMPASS Data.

Source: Jorge Salazar at TACC