In this video from the recent Argonne Training Program on Extreme-Scale Computing, Bill Gropp from the University of Illinois Urbana-Champaign presents: MPI and Hybrid Programming Models.
“Blue Waters supports a variety of programming models, including the “MPI everywhere” model that is the most common among today’s MPI applications. In addition, it will support a variety of other programming models. The programming models may be used instead of MPI or they may be used in combination with MPI. Such a combined programming model is often called a hybrid model. The most familiar of the models used in combination with MPI is OpenMP, which is designed for shared-memory systems and is based on the use of multiple threads in each MPI process. This programming model has found mixed success to date, with many experiments showing little benefit while others show promise. The reason for this is related to the use of OpenMP within MPI programs—where OpenMP is used to complement MPI, for example, by providing better support for load-balancing adaptive computations or sharing large data tables, it can provide a significant benefit. Where it is used as an alternative to MPI, OpenMP often has difficulty achieving the performance of MPI (MPI’s much-criticized requirement that the user directly manage data motion ensures that the programmer does in fact manage that memory motion, leading to improved performance). This suggests that other programming models can be productively combined with MPI as long as they complement, rather than replace, MPI.”
Download the Slides (PDF) * Sign up for our insideHPC Newsletter
