In this video, Maciej Besta and Torsten Hoefler describe their work on Slim Fly: A Cost Effective Low-Diameter Network Topology, which was honored as the Best Student Paper at SC14.
“HPC systems have kept growing in the last decades, producing vast amounts of data. Over time, we noticed how increasingly difficult it is to manage these data. Lustre as parallel filesystem is widely used in the HPC domain. It answers most of the requirements of the new generations of the HPC systems. Before Lustre 2.4, the metadata operation was managed by single metadata server and target. Such single operation became a kind of bottleneck. As a first step to help solving this problem Lustre multiple metadata server and targets (DNE) came out with Lustre 2.4. This talk will cover the first experience with Lustre DNE, showing some results and discussing the next step.”