In this video from the DDN User Group at SC14, Robert Triendl presents: Optimizing Lustre and GPFS Solutions with DDN.
“By working with ThinkParQ, we have been able to leverage one of the best and highest performance storage systems for scale-out deployment,” said Dr. Joseph Landman, CEO of Scalable Informatics. “When testing a write-dominated workload using fio, IOR, and io-bm,a single rack of FastPath Unison with BeeGFS running on spinning disks sustained in excess of 40GB/s for multi-terabyte sized writes,far outside of cache. This level of performance comes from the combination of FastPath Unison hardware design, the Scalable Informatics Operating System (SIOS), and the excellent BeeGFS filesystem.”
From Wall Street to the Great Wall, enterprises and institutions of all sizes are faced with the benefits – and challenges – promised by ‘Big Data’. But before users can take advantage of the near limitless potential locked within their data, they must have affordable, scalable and powerful software tools to manage the data.
This week we look at various attributes including how easy it is to scale Lustre file systems. The inherent scalability of Lustre aggregates storage capacity across many servers. I/O bandwidth also scales as more storage servers are added, and can be dynamically adjusted as needs change and demands for more storage capacity and bandwidth grow.
With the release of Intel Enterprise Edition (EE) for Lustre software, commercial customers have an opportunity to employ a production-ready version of Lustre optimized for business HPDA. Intel EE for Lustre includes the open source distribution of Lustre with the latest features, fully tested and supported by Intel, a major collaborator in the development of the Lustre parallel file system.
The Lustre file system is a open source, parallel file system that supports the requirements of leadership class HPC and Enterprise environments worldwide. Lustre provides a POSIX compliant interface and scales to thousands of clients, petabytes of storage, and has demonstrated over a terabyte per second of sustained I/O bandwidth. Many of the largest and most powerful supercomputers on Earth today are powered by the Lustre file system, including over 60% of the TOP100 sites.
The recent release of a commercial version of the Lustre* parallel file system was big news for business data centers facing ever expanding data analysis and storage demands. Now, Lustre, the predominant high-performing file system installed in most of the supercomputer installations around the world, could be deployed to business customers in a hardened, tested, easy to manage and fully supported distribution.