Lustre* is not just for the national labs any longer. It was born out of serving up data extremely fast to the world’s most powerful HPC clusters using parallel I/O to improve performance and scalability. Here are five reasons why Lustre is enterprise-ready.
“The combination of the ephemeral nature of the cloud and directly addressable archives such as S3 suggest novel methods for using the Lustre HSM interface. Persistent data sets in the cloud need to be managed independently from an ephemeral filesystem and compute resources. Managing datasets in the cloud could, for example, involves importing data from Amazon’s S3 back into a freshly-created Lustre filesystem, performing I/O intensive computations, and then persisting the datasets back to S3 before terminating the filesystem and compute resources. Alternatives for archive formats will also be discussed. AWS S3 will be used for concrete examples, but the general methods should be applicable to other cloud environments as well.”
In this video from the LAD’14 Conference in Reims, Daniel Rodwell from the National Computational Infrastructure (NCI) in Australia presents: Lustre HSM.
Today SGI announced the immediate availability DMF for Lustre. “With SGI DMF for Lustre, enterprises can ensure constant access to data while leveraging the significant cost benefits and scalability of tiered storage.”
Galen Shipman describes why Hierarchical Storage Management capabilities are such an important part of the new Lustre 2.5 release.