The NASA Center for Climate Simulation (NCCS) is boosting supercomputer capacity to over 3 Petaflops to meet the exploding demands of NASA’s Earth science modeling efforts. The upgraded Discover supercomputer from SGI will be powered by 64,512 current-generation 14-core Intel E5-2697v3 (Haswell) cores.
NCCS kept its user community’s needs constantly in mind when tuning the supercomputer architecture with SGI. “As model codes are scaling to higher and higher core counts, they need larger amounts of memory,” said NCCS High- Performance Computing Lead Daniel Duffy. Extending a practice begun with SCU9, all three SCUs have over 4 gigabytes of memory per core.
SCU10’s 138 terabytes of memory are proving handy for its primary user—the Downscaling Project. The NASA-wide investigation is assessing the credibility of downscaled climate projections, which use the results from global climate models to drive higher-resolution regional models. For this project the global model is GEOS-5 running at 12 kilometers (km), and the regional model is NU-WRF running at 24, 12, and 4 km. Scientists are comparing how well the models predict three weather phenomena impacting the continental United States: Northeast wintertime storms, mid-continent summertime storms, and West Coast wintertime atmospheric rivers.
Another computer architecture advantage for NASA modeling efforts is a fully non-blocking interconnect fabric. On the new SCUs each 28-core node can communicate directly with every other node via Fourteen Data Rate (FDR) InfiniBand rated at 56 gigabits per second. “The research runs for GEOS-5 are getting very large and require extremely fast computations and communications. This is a major challenge if the high-performance computing fabric is not suited to those runs,” Duffy said.
Since a high-resolution simulation can produce up to several petabytes of data, NCCS is more than doubling Discover’s online disk to 33 petabytes.
