Building Efficient HPC Clouds with MVAPICH2 and OpenStack over SR-IOV Enabled InfiniBand Clusters

Print Friendly, PDF & Email

DK Panda, Ohio State University

DK Panda, Ohio State University

In this video from OpenStack Summit in Austin, DK Panda from Ohio State University presents: Building Efficient HPC Clouds with MVAPICH2 and OpenStack over SR-IOV Enabled InfiniBand Clusters.

“Single Root I/O Virtualization (SR-IOV) technology has been steadily gaining momentum for high-performance interconnects such as InfiniBand. SR-IOV can deliver near native performance but lacks locality-aware communication support. This talk presents an efficient approach to build HPC clouds based on MVAPICH2 over OpenStack with SR-IOV. We discuss the high-performance design of virtual machine-aware MVAPICH2 library over OpenStack-based HPC Clouds with SR-IOV. A comprehensive performance evaluation with micro-benchmarks and HPC applications has been conducted on an experimental OpenStack-based HPC cloud and Amazon EC2. The evaluation results show that our design can deliver near bare-metal performance. The MVAPICH2 over OpenStack software package presented in this talk are publicly available.”

What can I expect to learn?
  • What are the performance benefits of SR-IOV and its limitations on locality-aware inter-VM communication within the same physical node?
  • How to design a high-performance MPI library to efficiently take advantage of novel features such as SR-IOV and IVShmem provided in HPC clouds?
  • How to build an HPC Cloud to deliver near-native performance for MPI applications over SR-IOV enabled InfiniBand clusters?
  • How much performance improvement can be achieved by our proposed design on MPI point-to-point operations, collective operations and applications in HPC clouds?

Dr. Dhabaleswar K. (DK) Panda is a Professor and University Distinguished Scholar of Computer Science and Engineering at the Ohio State University. He has published over 350 papers in major journals and international conferences. Prof. Panda and his research group members have been doing extensive research on modern networking technologies including InfiniBand, High-Speed Ethernet and RDMA over Converged Enhanced Ethernet (RoCE). The MVAPICH2 (High Performance MPI over InfiniBand, iWARP and RoCE) and MVAPICH2-X software libraries, developed by his research group (, are currently being used by more than 2,525 organizations worldwide (in 77 countries). This software has enabled several InfiniBand clusters to get into the latest TOP500 ranking during the last decade. More than 348,000 downloads of this software have taken place from the project’s website alone. The new RDMA-enabled Apache Hadoop, Spark, and Memcached packages, and OSU HiBD benchmarks (OHB) are publicly available from the High-Performance Big Data (HiBD) project site ( Prof. Panda’s research has been supported by funding from US National Science Foundation, US Department of Energy, and several industry including Intel, Cisco, Cray, SUN, Mellanox, QLogic, NVIDIA and NetApp. He is an IEEE Fellow and a member of ACM.

Sign up for our insideHPC Newsletter