Over at the MPI Blog, Jeff Squyres writes that the distance-from-home analogy is good way to help explain application latency.
So when you send a message to a peer (e.g,. MPI_SEND to another MPI process), consider with whom your communicating: are they next door, in the next subdivision, or in the next city? That gives you an idea of the magnitude of the cost of communicating with them. But let’s add another dimension here: caches and RAM. Data locality is a major factor in performance, and is frequently under-appreciated.
Read the Full Story.