In this video from the Ginormous System Conference, Dr. Garth Gibson from Panasas presents: Disruptive Change in Storage Technologies for Big Data. Gibson describes the evolution from terabytes to petabytes and on to the promise of exascale computing and how NAND flash is changing the rules along the way. Other topics include parallel file systems, object storage and the challenges of working with both very large and very small data sets in unified systems.
My story in a nutshell is that as things get larger, if they get larger and we operate them on larger sizes, we actually have pretty good technology for dealing with size. We suffer primarily from scale and the number of components that can fail, and keeping consistency on those. The consistency issue is a serious one for storage systems that are always available.