In this video from LUG 2016, Olaf Weber from SGI presents: Multi-rail LNet for Lustre.
“With the Lustre parallel filesystem being adapted widely among HPC centers, SGI wanted to create the ability to couple the SGI UV platform into Lustre with the file system bandwidth to match the SGI UV in-memory processing capability. Currently, Lustre only supports a single network connection for a compute node into the parallel file system. This is fine when connecting to the typical dual-socket nodes of most scale-out HPC systems, but with the SGI UV supporting up to 256 sockets and up to 64TB of memory in a single node, it looks more like a straw. In collaboration with Intel, SGI set about creating support for multiple network connections to the Lustre filesystem, with multi-rail support. With Intel Omni-Path and EDR Infiniband driving to 200Gb/s or 25GB/s per connection, this capability will make it possible to start moving data between a single SGI UV node and the Lustre file system at over 100GB/s.