Today SC15 announced the winners of their annual Test of Time Award. The Award recognizes an outstanding paper that has deeply influenced the HPC discipline.
Entitled “The NAS Parallel Benchmarks – Summary and Preliminary Results,” the winning paper was written by D. Bailey, E. Barszcz, J. Barton, D. Browning, R. Carter, L. Dagum, R. Fatoohi, P. Frederickson, T. Lasinski, R. Schreiber, H. Simon, V. Venkatakrishnan, and S. Weeratunga from the SC91 conference.
The paper and benchmark captures specifications and implementations of an important set of representative scientific codes,” said Jack Dongarra, SC15 Test of Time Award Chair from University of Tennessee, Knoxville. “The work is still actively used, and has inspired numerous sets of benchmarking codes that continue to drive research and development innovation.”
In 1991, this team of computer scientists from the Numerical Aerodynamic Simulation Program—predecessor to the NASA Advanced Supercomputing (NAS) facility at Ames Research Center—unveiled the NAS Parallel Benchmarks (NPB), developed in response to the U.S. space agency’s increasing involvement with massively-parallel architectures and the need for a more rational procedure to select supercomputers to support agency missions. Then, existing benchmarks were usually specialized for vector computers, with shortfalls including parallelism-impeding tuning restrictions and insufficient problem sizes, making them inappropriate for highly-parallel systems.
The NPBs mimic computation and data movement of large-scale computational fluid dynamics (CFD) applications, and provide objective evaluation of parallel HPC architectures. The original NPBs featured “pencil-and-paper” specifications, which bypassed many difficulties associated with standard benchmarking methods for sequential or vector systems. The principal focus was in computational aerophysics, although most of these benchmarks have broader relevance for many real-world scientific computing applications.
The NPBs quickly became an industry standard and have since been implemented in many modern programming paradigms. Since 1991, research areas influenced by the NPBs have broadened to include network design, programming languages, compilers, and tools. Google Scholar yields over 27,000 results for the NBPs, with about 7,400 citations since 2014.
Today’s version is alive and well, and continues to significantly influence NASA projects. It is used around the world by national labs, universities, and computer vendors to evaluate sustained performance of highly-parallel supercomputers and the capability of parallel compilers and tools.