The Lustre Parallel File System—A Landscape of Topics and Insight from the Community

Since its beginnings in 1999 as a project at Carnegie Mellon University, Lustre, the high performance parallel file system, has come a long, long way. Designed and always focusing on performance and scalability, it is now part of nearly every High Performance Computing (HPC) cluster on the Top500.org’s list of fastest
computers in the world—present in 70 percent of the top 100 and nine out of the top ten. That’s an achievement for any developer—or community of developers, in the case of Lustre—to be proud of. Learn what the HPC Community is saying about Lustre.

SGI Provides Total with Improved Modeling to Support Decision Making

Total, one of the largest integrated oil and gas companies in the world, announced they are boosting the compute power of their SGI Pangea supercomputer with an additional 4.4 petaflops provided by a new SGI ICE X system and based on the Intel Xeon processor. Purchased last year, the new SGI system is now in production and will allow Total to determine the optimal extraction methods more quickly. The SGI supercomputer allows Total to improve complex modeling of the subsurface and to simulate the behavior of reservoirs, reducing the time and costs associated with discovering and extracting energy reserves.

Lustre: This is Not Your Grandmother’s (or Grandfather’s) Parallel File System

“Over the last several years, an enormous amount of development effort has gone into Lustre to address users’ enterprise-related requests. Their work is not only keeping Lustre extremely fast (the Spider II storage system at the Oak Ridge Leadership Computing Facility (OLCF) that supports OLCF’s Titan supercomputer delivers 1 TB/s ; and Data Oasis, supporting the Comet supercomputer at the San Diego Supercomputing Center (SDSC) supports thousands of users with 300GB/s throughput) but also making it an enterprise-class parallel file system that has since been deployed for many mission-critical applications, such as seismic processing and analysis, regional climate and weather modeling, and banking.”

DDN Sets World Record STAC Performance

Today DDN announced record performance on the Securities Technology Analysis Center (STAC) benchmark. Using the company’s EXAScaler storage solution, DDN set new public records for multiple workload types and sizes, including large and small workloads as well as I/O and compute-intensive workloads.

Lustre* For the Enterprise

Lustre* is not just for the national labs any longer. It was born out of serving up data extremely fast to the world’s most powerful HPC clusters using parallel I/O to improve performance and scalability. Here are five reasons why Lustre is enterprise-ready.

HPC News Bytes for June 19, 2015

While we’re always on the lookout for HPC news, not everything makes it to the front page. Notable items from this week include big boosts for Apache Spark, Containerization, and Lustre.

Interview: Intel Taking Lustre into New Markets

“It’s been nearly three years since Intel acquired Whamcloud and its Lustre engineering team. With Intel’s recent announcement that Lustre will power the 2018 Aurora supercomputer at Argonne, we took the opportunity to catch up with Brent Gorda, general manager of Intel High Performance Data Division at Intel Corporation.”

Deploying Hadoop on Lustre Storage: Lessons Learned and Best Practices

In this video from LUG 2015 in Denver, J.Mario Gallegos from Dell presents: Deploying Hadoop on Lustre Storage: Lessons Learned and Best Practices. “Merging of strengths of both technologies to solve big data problems permits harvesting the power of HPC clusters on very fast storage.”