In this special guest feature, Ken Strandberg offers this live report from Day 2 of the Lustre User Group meeting in Portland. “Scott Yockel from Harvard University shared how they are deploying Lustre across their massive three data centers up to 90 miles apart with 25 PB of storage, about half of which is Lustre. They’re using Docker containers and employing a backup strategy across the miles of every NFS system, parsing of the entire MDT, and includes 10k directories of small files.”
“The Lustre User Group (LUG) 2016 conference is well under way. The morning of the first day was spent on looking at Lustre today and tomorrow and security developments in the code. Peter Jones and Andreas Dilger described what is in the newest release of Lustre 2.8 and will be in Lustre 2.9, targeted for release this fall, and beyond. These features include growing support for ZFS, security, multi-rail LNET, progressive file layouts, project quotas, and more.”
“With Supermicro’s 90 top-load 3.5” hot-swap bay JBOD as the storage core of our Lustre Pod Cluster, we maximize performance, density and capacity and simplify serviceability for massive scale HA storage deployments. Combining our preconfigured, validated 2U SuperStorage OSS, 1U Ultra SuperServer with Intel Enterprise Edition for Lustre software, and global service and support, Supermicro has the Total Solution for Lustre ready for HPC, Genomics and Big Data.”
Today Seagate announced it will incorporate Intel Enterprise Edition for Lustre (IEEL), a big data software platform, into its market-leading ClusterStor storage architecture for high-performance computing. The move will strengthen Seagate’s HPC data storage product line and provide customers with an additional choice of Lustre parallel file systems to help drive advancements in the HPC and big data market.
“Traditionally, storage have been using brute force rather than intelligent design to deliver the required throughputs but the current trend is to design balanced systems with full utilization of the back-end storage and other related components. These new systems need to use fine grained power control all the way down to individual disk drives as well as tools for continuous monitoring and management of these systems. In addition, the storage solutions of tomorrow needs to support multiple tiers including backend archiving systems supported by HSM as well multiple file systems if required. This presentation is intended to provide a short update of where Seagate HPC storage is today.”
As high performance and webscale applications become mainstream, HPE’s continued focus on this market is yielding positive results for our customers,” said Bill Mannel, vice president and general manager, HPC, Big Data and IoT Servers, HPE. “Already, more than a third of the HPC market is using HPE compute platforms to enhance scientific and business innovation and gain a competitive edge. Today’s announcement reinforces our commitment to delivering new infrastructure solutions that satisfy our customers’ insatiable need for massive compute power to fuel new applications and unlock the value of their data.”
Since its beginnings in 1999 as a project at Carnegie Mellon University, Lustre, the high performance parallel file system, has come a long, long way. Designed and always focusing on performance and scalability, it is now part of nearly every High Performance Computing (HPC) cluster on the Top500.org’s list of fastest
computers in the world—present in 70 percent of the top 100 and nine out of the top ten. That’s an achievement for any developer—or community of developers, in the case of Lustre—to be proud of. Learn what the HPC Community is saying about Lustre.
GDH Consulting in Albuquerque is seeking a Lustre File Systems HPC Software Integrator Integrator in our Job of the Week. “A systems programmer with proficient coding experience in C is required to participate in the implementation of research file systems and related systems software. The qualified candidate will have the opportunity to work on innovative approaches to data movement on large scale Linux clusters, and to develop new strategies for attacking the challenging issues in the I/O arena which arise from the requirements of the file system to scale and be robust.”
Today the Open Scalable File Systems (OpenSFS) community announced the release of Lustre 2.8.0, the fastest and most scalable parallel file system. OpenSFS, founded in 2010 to advance Lustre development, is the premier non-profit organization promoting the use of Lustre and advancing its capabilities through coordinated releases of the Lustre file system.
In this podcast, Rich Brueckner interviews Hugo Saleh, Director of Marketing for the Intel High Performance Computing Platform Group. They discuss the new Intel® Xeon® processor E5-2600 v4 product family, based upon the Broadwell microarchitecture, and the first processor within Intel® Scalable System Framework (Intel® SSF). Hugo describes how the new processors improve HPC performance and examine the impact of Intel® SSF on vastly different procurements ranging from the massive 200 Petaflops Aurora system to small and large enterprise clusters plus scientific systems.