“Intel supports users, system integrators, and OEMs using ZFS with Intel Lustre. In this presentation, we summarize the results of proof-of-concept (PoC) on a variety of the ZFS configurations. We cover sequential and metadata performance, data Integrity, manageability, availability and reliability. The work identifies the areas where development should be focused in order to fill gap in performance or functionality and encourage system administrator to integrate this technology with the existing high availability framework like Pacemaker/Corosync. We also cover the most important tunables for ZFS in combination with Lustre and the most notable metrics for Lustre and ZFS.”
In this video from the LAD’14 Conference in Reims, Gabriele Paciucci from Intel presents: Current Status of the Adoption of ZFS as Backend File System for Lustre. “With the release of Lustre 2.4 support for ZFS Lustre servers has arrived. Historically, Lustre has only supported ext4/ldiskfs servers and while those servers have performed well they do suffer from a number of well known limitations. Extending Lustre to use a next generation filesystem like ZFS allows it to achieve greater levels of scalability and introduces new functionality.”
“Sharing a common code base with other Open ZFS platforms has given ZFS on Linux the opportunity to rapidly implement features available on other Open ZFS platforms. At present, Illumos is the reference platform in the Open ZFS community and despite its ZFS driver having hundreds of features, ZoL is only behind on about 18 of them.”
This week the Joint Institute for Computational Sciences at ORNL announced a partnership with WARP Mechanics to study next generation object storage technologies for use in HPC environments. In this video from LUG 2014, Josh Judd from Warp Mechanics presents: Practical Applications of Lustre/ZFS Hybrid Systems.”
“LLNL’s largest supercomputer is paired with a 55-petabyte file system, known as Grove, that stores vast quantities of simulation data. Grove must transfer information to and from Sequoia at a minimum of 500 gigabytes per second. To support Grove’s high storage capacity and bandwidth requirements, LC software developers have engaged in a multi-year project to replace much of Grove’s Lustre foundation with ZFS. “
Josh Judd from Warp Mechanics describes how the company delivers Lustre Over ZFS on Linux. “No single technology solves all problems faced in today’s complex world. WARP Mechanics’ philosophy is to customize the many and varied systems into the exact set of solutions required to address the problems. WARP Mechanics leverages tried–and-true technologies from the most advanced systems and removes the complexity, delivering customized turnkey solutions.”
With the release of Lustre 2.4 support for ZFS Lustre servers has arrived. Historically, Lustre has only supported ext4/ldiskfs servers and while those servers have performed well they do suffer from a number of well known limitations. Extending Lustre to use a next generation filesystem like ZFS allows it to achieve greater levels of scalability and introduces new functionality.