MPI Performance Tradeoffs: Eager RDMA

Print Friendly, PDF & Email

Cisco’s Jeff Squyres discusses MPI trade-offs between performance and resource consumption.

Eager RDMA: This is an optimization that came out of the realization that RDMA operations on certain networks (e.g., iWARP, IBoE [a.k.a. RoCE], and InfiniBand) are typically faster than send/receive operations.  The idea is simple: use RDMA to send short, unexpected messages instead of send/receive semantics.  Essentially, this means that a short message will just magically show up in the receiver’s memory without the receiving process doing anything — the RDMA transfer is completely handled in hardware.

The receiver can be aggressively polling on a specific location in memory to find out that a new message has arrived.  Polling in this manner can notice the new message much faster than, say, an interrupt, or some other OS-induced mechanism.

Read the Full Story.