Today Seagate announced at SC14 a new version of its ClusterStor Engineered Solution for Lustre that adds new features to the Lustre parallel data storage file system used in HPC environments and the ClusterStor Hadoop Workflow Accelerator, a set of Hadoop tools, services and support for HPC environments.
Seagate ClusterStor Engineered Solutions for Lustre adds important new features to the Lustre parallel data storage file system, scaling from small workgroup clusters to large-scale computing clusters requiring storage support of up to 1TB/sec performance and up to 100s of PB storage capacity from a single file system. Updated features include scalability advancements to support larger computing clusters, improved security capabilities including government security compliance and ease of deployment for upcoming Lustre releases.
We’re excited to implement Seagate’s latest version of the ClusterStor Engineered Solution for Lustre as part of our Cray Sonexion 2000 offering. We believe this will build upon and expand our already successful and productive partnership,” said Barry C. Bolding, Vice President, Marketing and Business Development, Cray Inc. “The emphasis Seagate is placing on HPC solutions means continued innovation for customers requiring data intensive computing and provides a solid framework for Cray’s Lustre parallel file system storage offering.”
Engineered for high-performance computing, Seagate’s HPC solution combines high-density storage enclosures, the operating system, hardware controllers, and the Lustre file system in a consolidated, scale-out high performance-computing storage platform. The award-winning Seagate HPC solution removes the complexities associated with deploying and maintaining traditional high-performance systems yielding faster time to results, ease of use, and world-class performance to meet mission-critical needs,” said Ken Claffey, Vice President of ClusterStor, Seagate Cloud Systems and Solutions. “The increased scalability, ease-of-management and improvements to Seagate ClusterStor Engineered Solutions for Lustre will be welcomed by our OEM partners and end users. We look forward to sharing these advancements with Seagate’s partners as well as our public and private sector communities during the Super Computing (SC14) conference in New Orleans.”
The availability of the ClusterStor™ Hadoop Workflow Accelerator provides the tools, services, and support for High Performance Computing (HPC) customers who need the best performing storage systems for Big Data Analytics. The Hadoop Workflow Accelerator is a set of Hadoop optimization tools, services and support that leverages and enhances the performance of ClusterStor™, the market leading scale-out storage system designed for Big Data analysis. Computationally intensive High Performance Data Analytics (HPDA) environments will benefit from significant reductions in data transfer time with the Hadoop Workflow Accelerator. This solution also includes the Hadoop on Lustre Connector, which allows both Hadoop and HPC Lustre clusters to use exactly the same data without having to move the data between file systems or storage devices.
Data-intensive computing has long been a part of HPC, but newer analytical approaches using Hadoop and other methods, such as graph analytics, will help drive strong growth in high performance data analysis, which is the market for Big Data needing HPC. The Hadoop Workflow Accelerator is designed to serve both the technical computing and commercial sides of this converging Big Data-HPC market that IDC forecasts will exceed $4 billion in 2018,” said Steve Conway, IDC Research Vice President, High Performance Computing.”
The Hadoop Workflow Accelerator supports Hadoop distributions based on Open Source Apache Hadoop. Seagate is working with leading Hadoop distributors to offer best-in-class solutions to HPC customers and will provide tighter integration between the Hadoop Workflow Accelerator and other Hadoop distributions in future releases.
Organizations not only want to manage the tremendous volume of data that they are collecting from a wide variety of sources, they also want to derive new insights that enable actionable intelligence and improve operational efficiency,”said Ken Claffey, Vice President of ClusterStor, Seagate Cloud Systems and Solutions. “Seagate’s award-winning ClusterStor scale-out HPC solutions, now with our Hadoop Workflow Accelerator options, enable organizations to optimize Big Data workflows and centralize data storage for High Performance Data Analytics solutions. TeraSort benchmark results have the Hadoop Workflow Accelerator outperforming Hadoop on the Hadoop Distributed File System (HDFS) by 38% on the same hardware. The Hadoop Workflow Accelerator meets our customer’s performance demands and optimizes the performance of Hadoop Ecosystem deployments, thus helping customers achieve the fastest time to results for their data intensive workloads and hardware configuration.”
The Seagate ClusterStor systems’ innovative scale-out HPC architecture enables a central repository allowing both HPC and Hadoop analytics tools to be run simultaneously on the same data sets in ClusterStor. The Hadoop Workflow Accelerator reduces time to results by enabling immediate Hadoop data processing from the start of each job, and eliminates the time consuming step of bulk copying large amounts of data from a separate data repository. Hadoop environments can now scale computing and storage resources independently, optimizing analysis resources, while supporting centralized high-performance data repositories of 100’s of PBs of storage capacity.
Seagate will demonstrate the Hadoop Workflow accelerator at SC14.
Visit Seagate booth #3239 at SC14.
