In scale-out NAS, HPC storage performance can be driven by several factors:
- Elimination of file processing bottlenecks
- Parallel data paths
- Fast storage components
- Load balancing
- Linearly scalable architecture
- Flash acceleration
In legacy NAS systems, all data flows through the NAS heads in-band, which can cause file processing bottlenecks. Performance can be substantially enhanced when file metadata is processed outside the data path. Parallel data paths can also help eliminate bottlenecks as much higher overall data transfer speed can be achieved.
This is the third article in a series on HPC Storage
Fast storage components that incorporate both flash as well as hard disk drives, with a large cache contribute to accelerating all workloads. An intelligent architecture can ensure that the right files reside in the right place. For example, data can be placed on the appropriate media based on file size. Small IOPs oriented files can be read and written to SSDs for exceptionally fast I/O performance while also using the SSD to hold metadata for better file system responsiveness. Large sequential file transfers are done on the most cost effective media which are hard disk drives and can leverage the low cost per GB which is characteristic of large capacity drives. This method leverages the acceleration of flash while retaining the economics of hard drives.
Automatic load balancing eliminates hot spots by distributing workloads among all nodes so that the load on each node is roughly equal. As nodes are added so is bandwidth and capacity and workloads can be redistributed.
Maintaining performance at scale also requires that rebuild times be consistent at scale. Drive rebuild times can even worsen at scale in lesser scale-out architectures.
This linear performance scaling eliminates the multiple islands of storage that are so prevalent with scale-up architectures.
Next week we’ll dive into high availability storage. If you prefer you can download the complete insideHPC Guide to HPC Storage in PDF format courtesy of Panasas.
