In this video from the GPU Technology Conference, Chris Gottbrath from Rogue Wave Software presents: Three Ways to Debug Parallel CUDA Applications: Interactive, Batch, and Corefile.
There are now three ways that one can approach debugging an MPI application running on a cluster that is accelerated with NVIDIA GPUs. These are interactive hands on debugging, non-interactive batch debugging and post-mortem debugging using the GPU corefile functionality introduced in CUDA 7.0. Each of these techniques is a useful tool with benefits and limitations. This talk will introduce these three debugging techniques and provide some suggestions on selecting the optimal approach for a variety of debugging scenarios such as hangs, numerical errors, and crashes. Specific examples will be given using the TotalView debugger but the concepts covered may apply to other debugging tools such as GDB and the NVIDIA NSIGHT debugger.”