InsideHPC Guide to HPC Storage

Print Friendly, PDF & Email

This the first article in series from the editors of insideHPC on HPC storage sponsored by Panasas.

Introduction to HPC Storage

HPC and technical computing environments require the collection, storage, and transmission of large-scale datasets. To meet these demands, datacenter architects must consider how increasing storage capacity over time will affect HPC workload, performance, and system availability.

Guide to HPC Storage - coverWhile many enterprises have looked to scale-up NAS to meet their storage needs, this approach can lead to data islands that make it difficult to share data. Distributed, scale-out storage was developed to get around the technology limitations of scale-up NAS architectures.

A traditional scale-up NAS system has a limited fixed amount of controllers, cache, and drive slots. When it reaches capacity, the customer must either upgrade to a larger system, or acquire another system. Alternatively, organizations also acquire storage platforms that are larger than current requirements to ensure bandwidth and disk space is available for future expansion. If expansion does not occur within the near future, or, is less than capacity forecasts, the resources provided by the large platform are simply wasted – and at high cost. As the amount of unstructured data grows to drive innovation and discovery, organizations are learning that new approaches, such as scale-out NAS, are needed to manage the data for both efficiency and scale.

Scale-out NAS systems are capable of almost limitless expansion, and can be managed as a single storage resource. Well-executed scale-out storage has three main benefits:

  • Balanced and efficient scaling of capacity and performance
  • Aggregation of multiple cost-effective nodes into a single large common pool of storage
  • Architecture for high performance across a variety of file workloads

The scale-out methodology creates an environment that supports data growth on an as-needed basis.  Scale-out storage architecture uses multiple nodes to scale both capacity and performance and to create a common storage pool. The approach is to add additional nodes as required that work in tandem with each other.  This approach is also lower in cost, as additional nodes need not be purchased until required. When a node reaches its capacity, one simply purchases another, and the system can grow in response to the needs of an organization.

The ideal scale-out NAS system will have:

  • Exceptional aggregate performance at scale
  • Linear and independent scalability of performance with capacity
  • Leading reliability and availability at scale
  • Ease of management regardless of scale

Over next 5 weeks we look at many aspects of a well designed HPC storage environment including:

If you prefer you can download the complete insideHPC Guide to HPC Storage in PDF format courtesy of Panasas.