Sign up for our newsletter and get the latest HPC news and analysis.

Introduction to the Lustre File System

lustre-logo2

Although there are a number of truly huge implementations of Lustre today, the community is still far from reaching the maximum configurations that the Lustre architecture is designed for. Inside the Lustre File System describes the basics of how the Lustre File System operates with descriptions of the newest features.

The State of the Lustre Community

Brent Gorda, GM, Intel High Performance Data Division

Another year has passed and, with it, more growth for Lustre and the community around it. This year, as well as welcoming many new HPC sites to the community, we also delight in the evidence of activity from non-traditional Lustre sites and users.

Inside Lustre Hierarchical Storage Management (HSM)

Hierarchical Storage Management

There is always different levels of importance assigned to various data files in a computer system, specifically a very large system that is storing petabytes of data. In order to maximize the use of the highest speed storage, Hierarchical Storage Management (HSM) was developed to move and store data within easy use of users, yet at the appropriate speed and price.

Lustre 101: Inside The Lustre File System

lustre-logo-PMS

The white paper, Inside the Lustre File System, describes the inner workings of Lustre in a way that is easy to understand, yet is technical enough for many users and systems administrators. Lustre is a mature and stable file system that has consistently been able to respond to the needs of organizations that require high performance throughput and expanding capacity.

Video: Optimizing Lustre and GPFS Solutions with DDN

ddn

In this video from the DDN User Group at SC14, Robert Triendl presents: Optimizing Lustre and GPFS Solutions with DDN.

Video: Scalable Informatics Steps Up IO at SC14

joe

“By working with ThinkParQ, we have been able to leverage one of the best and highest performance storage systems for scale-out deployment,” said Dr. Joseph Landman, CEO of Scalable Informatics. “When testing a write-dominated workload using fio, IOR, and io-bm,a single rack of FastPath Unison with BeeGFS running on spinning disks sustained in excess of 40GB/s for multi-terabyte sized writes,far outside of cache. This level of performance comes from the combination of FastPath Unison hardware design, the Scalable Informatics Operating System (SIOS), and the excellent BeeGFS filesystem.”

Unleashing the Power of Parallel Storage

Hadoop Cluster

From Wall Street to the Great Wall, enterprises and institutions of all sizes are faced with the benefits – and challenges – promised by ‘Big Data’. But before users can take advantage of the near limitless potential locked within their data, they must have affordable, scalable and powerful software tools to manage the data.

Easily Scale Lustre to Petabytes of Data

lustre-logo2

This week we look at various attributes including how easy it is to scale Lustre file systems. The inherent scalability of Lustre aggregates storage capacity across many servers. I/O bandwidth also scales as more storage servers are added, and can be dynamically adjusted as needs change and demands for more storage capacity and bandwidth grow.

Commercial Grade Lustre File Systems

commercial grade lustre sm

With the release of Intel Enterprise Edition (EE) for Lustre software, commercial customers have an opportunity to employ a production-ready version of Lustre optimized for business HPDA. Intel EE for Lustre includes the open source distribution of Lustre with the latest features, fully tested and supported by Intel, a major collaborator in the development of the Lustre parallel file system.

What is Lustre?

lustre

Fifteen years ago, the HPC community began developing a super fast and highly scalable parallel file system in response to the limitations of NFS. Called Lustre, it is the dominant file system technology now in use at half of the top 100 supercomputer installations worldwide (top500.org).