Given that MPI is so wide used in high-performance technical computing, there are a number of software packages available to help programmers develop better code. The first among these is the debugger. There used to be a few open source packages available, but they have all seemingly disappeared.
So for commercial debuggers, the most common one today is TotalView from Etnus, which can also be used to trace multi-threaded code. Green Hills’s TimeMachine is a debugger that allows the user to step forwards and backwards through execution time to pin-point the fault. And Allinea’s Distributed Debugging Tool checks not only for memory leaks, but also for errors in using the C++ STL. In total, there are more than enough choices in the commercial space for picking an MPI debugger.
Once the code is running correctly, the user will want the code to run quickly, and for this there is the profiler. Unfortunately, there are not many products in this space as even the open source packages are no longer supported.
Furthermore, most of the commercial implementations have been withdrawn from the market. It appears that the total number of technical customers willing to pay for a multi-process debugger is quite small. Still, there is Allinea’s Optimization and Profiling Tool, which sports a timeline that shows mismatches in two-sided or collective communications, thereby allowing the user to restructure his code more accurately. Otherwise though, pickings are slim.
MPI does have support for profiling and debugging built into the standard. In addition to the usual MPI_ functions, there are an equivalent set of PMPI_ functions. In reality, PMPI is the implementation and MPI is the wrapper. Thus, one may develop a rather portable profiler or debugger by redefining the MPI functions to log calls and then invoke PMPI. Indeed, this is how Allinea’s products are developed.
One final program of note is SKaMPI, a benchmark tool for MPI implementations. The goal here is that developers can tweak their message-passing code to find the most optimum implementation for end-users.
