The performance-savvy HPC developer is in high demand today. Leaps in intra-node parallelism, memory performance and capacity are set to meet applications struggling to exploit existing systems head-on.
Nathan Rutman from Seagate presented this talk at the LAD’15 Conference. “So why is a spinning disk company talking about Flash? Last year, Seagate acquired Avago LSI’s flash division. We now have an array of flash-based storage. So I have nothing against Flash. This presentation is really on: Where does Flash make sense? I also have a personal agenda because I hate the term “Burst Buffer.” Everyone says “Burst Buffer” instead of saying “Flash.” It drives me crazy. So I’m going to explain what a Burst Buffer is and what it is not.”
For companies looking to test the viability of engineering in the cloud, Altair has teamed with Intel and Amazon Web Services (AWS) to offer an “HPC Challenge” for product design. In a nutshell, the program provides free cycles on AWS for up to 60 days, where users can run compute-intensive jobs for computer-aided engineering (CAE).
“Argonne National Laboratory is one of the laboratories helping to lead the exascale push for the nation with the DOE. We lead in a numbers of areas with software and storage systems and applied math. And we’re really focusing, our expertise is focusing on those new ideas, those novel new things that will allow us to sort of leapfrog the standard slow evolution of technology and get something further out ahead, three years, five years out ahead. And that’s where our research is focused.”
The Intel Omni-Path Architecture (Intel® OPA) whitepaper goes through the multitude of improvements that Intel OPA technology provides to the HPC community. In particular, HPC readers will appreciate how collective operations can be optimized based on message size, collective communicator size and topology using the point-to-point send and receive primitives.
“Researchers at the U.S. Department of Energy’s Argonne National Laboratory will be testing the limits of computing horsepower this year with a new simulation project from the Virtual Engine Research Institute and Fuels Initiative (VERIFI) that will harness 60 million computer core hours to dispel those uncertainties and pave the way to more effective engine simulations.”
Many Universities, private research labs and government research agencies have begun using High Performance Computing (HPC) servers, compute accelerators and flash storage arrays to accelerate a wide array of research among disciplines in math, science and engineering. These labs utilize GPUs for parallel processing and flash memory for storing large datasets. Many universities have HPC labs that are available for students and researchers to share resources in order to analyze and store vast amounts of data more quickly.
Companies already using High-performance Computing (HPC) with a Lustre file system for simulations, such as those in the financial, oil and gas, and manufacturing sectors, want to convert some of their HPC cycles to Big Data analytics. This puts Lustre at the core of the convergence of Big Data and HPC.