HPC systems have kept growing in the last decades, producing vast amounts of data. Over time, we noticed how increasingly difficult it is to manage these data. Lustre as parallel filesystem is widely used in the HPC domain. It answers most of the requirements of the new generations of the HPC systems. Before Lustre 2.4, the metadata operation was managed by single metadata server and target. Such single operation became a kind of bottleneck. As a first step to help solving this problem Lustre multiple metadata server and targets (DNE) came out with Lustre 2.4. This talk will cover the first experience with Lustre DNE, showing some results and discussing the next step.”
Get the InsideHPC Guide to Lustre from the editors of insideHPC, courtesy of Intel.