Sign up for our newsletter and get the latest HPC news and analysis.

Is Key-Value Data Storage Coming to a File System Near You?

Jeff Layton

As an example of what you can do with key-value storage and how simple it can be, Seagate has created a new storage drive called Kinetic that you address using REST-like commands such as get, put, and delete. A simple open-source library allows you to then develop IO libraries so that applications can perform IO to/from the drives. Some object storage solutions such as Swift have already been ported to use the Kinetic drives. Ceph is also developing a version that can use Kinetic drives. Other object based storage systems such as Lustre and Gluster could theoretically use this technology as well.”

Sponsored Post: Intel Cloud Edition Available for Lustre Software

lustre

“HPC cluster performance is often degraded because more and more data and larger files overwhelm limited hard drive capacity. But if you use Amazon Web Services (AWS), such bottlenecks may be a thing of the past. Intel, in collaboration with AWS, offers a Cloud Edition for Lustre Software that allows customers to use the power of the worlds’ most popular HPC storage system to increase scalability and performance.”

Sponsored Post: Intel Enterprise Edition Software for Lustre

lustre2

Intel Enterprise Edition Software for Lustre brings high performance storage to data-intensive applications. As a parallel file system, it lets managers reach their business goals sooner and at lower cost using the scalability of Lustre.

Case Study: Designing a High Performance Lustre Storage System

oss

Intel’s White Paper, “Architecting a High-Performance Storage System,” shows you the step-by-step process in the design of a Lustre file system. “Because performance is limited by the slowest component, the design methodology uses a pipeline approach to select and review each part, making sure the system requirements are met. By starting with the backend disk storage, gradually working up the pipeline to the client and employing an iterative design method, the paper show you how a Lustre file system is created.”

New Whitepaper: File I/O on Intel Xeon Phi (RAM disks, VirtIO, NFS, and Lustre)

colfax

“This paper provides information and benchmarks necessary to make the choice of the best file system for a given application from a number of the available options: RAM disks, virtualized local hard drives, and distributed storage shared with NFS or Lustre. We report benchmarks of I/O performance and parallel scalability on Intel Xeon Phi coprocessors, strengths and limitations of each option.”

Building the knoSYS 100 Genome Supercomputer

knoSYS_Church_v2

Over at Bio-IT World, Michael Fein from Silicon Mechanics describes how his company worked with Knome to design and build the first-of-its-kind knoSYS 100 Genome Supercomputer.

LAD’14 Issues Call for Participation

LAD14_Logo_web

EOFS and OpenSFS have issued their Call for Participation in the LAD’14 event, which will take place Sept. 23-24 in Reims, France.

Intel’s Eric Barton on the Need to Move Beyond Posix for Exascale IO

Eric Barton

In this video from ISC’14, Eric Barton from Intel describes the goals of the two-year FastForward Storage and IP Project, which the company wrapped up recently.

Xyratex/Seagate Update from ISC’14

torben2

In this video from ISC’14, Torben Kling-Petersen describes the latest technology news from Xyratex. The company’s ClusterStor technology for Lustre storage has just landed a major win with Bull at DKRZ.

Intel Enterprise Edition for Lustre Software at ISC’14

Brent Gorda, Intel

“The Intel Enterprise Edition for Lustre software unleashes the performance and scalability of the Lustre parallel file system for HPC workloads, including technical ‘big data’ applications common within today’s enterprises. It allows end-users that need the benefits of large–scale, high bandwidth storage to tap the power and scalability of Lustre, with the simplified installation, configuration, and management features provided by Intel Manager for Lustre software, a management solution purpose-built by the Lustre experts at Intel for the Lustre file system.”