Lustre 2.2 is out today, and Whamcloud’s Bryon Neitzel writes that it’s new Lustre Parallel Directory does a lot to improve performance.
Until now in Lustre, filename lookup and file system modifying operations (such as create and unlink) were protected by a single lock for an entire directory, thus limiting file writes to the directory to serialized access. Whamcloud has eliminated this bottleneck by introducing a parallel locking mechanism for the entire directory. This capability, called parallel directory operations (PDO), enables multiple metadata service threads to concurrently perform lookup, create, and unlink operations on a single directory. Multiple Object Indexes (MOIs) have also been implemented with PDO. These allow parallel lookups of a files inode from the File Identifier (FID), which is how a client identifies a unique file in the Lustre file system.