Our favorite Storage pundit Henry Newman has posted an interesting opinion piece calling for open standards for parallel I/O:
Parallel applications can do I/O in two different ways: They can write to a parallel file system such as GPFS, Lustre or Pan-FS, or they can write to a local file system on each server node in the configuration. The problem is that there is a dirty little secret: We have no standards other than the Message Passing Interface (MPI) for parallel processing communications and MPI-IO for parallel I/O. And I do not see any changes in sight because, in my opinion, the vendors that now control the standards bodies do not want to change anything as they can sell proprietary solutions that cost more in the long run, and lock you into that vendor. And it costs much more if you want to change vendors.
Newman goes on to propose that the HPC community should develop a parallel I/O standard for multiple languages that supports all of the features and functions of MPI-IO and more. That certainly won’t be easy, but I for one believe that it’s worth the effort.
