Sign up for our newsletter and get the latest HPC news and analysis.

Henry Newman on File System Interface Futures

Over at Enterprise Storage Forum, Henry Newman looks at the future of file systems and examines whether REST will overtake POSIX as an interface of choice for all applications.

We do not have a lot of POSIX file systems that scale today to 10s of PB and billions of files. There are three file systems in production with a parallel namespace (Gluster, PAN-FS, Lustre, and GPFS) and a new entry called Ceph. Ceph, GPFS Lustre and Pan-FS support parallel I/O, which is I/O from multiple threads (these threads could be running on multiple nodes) to a single file, but Gluster does not. On the other side there are dozens of vendors developing REST- and SOAP-based object management interfaces. Vendors are trying to create systems that support billions of objects in a single namespace. Given that the vendors are not constrained by the POSIX atomicity requirements and support for parallel I/O, this is far easier than developing this support inside a POSIX file system.

Read the Full Story.

Resource Links: