Where Did I Put That? Unexpected Linux Memory Migration

Cisco’s Jeff Squyres writes that HPC programmers may be subject to unexpected memory Linux migration:

I learned something disturbing earlier this week: if you allocate memory in Linux to a particular NUMA location and then that memory is paged out, it will lose that memory binding when it is paged back it.

Squyres goes on to say that memory locality is more important than ever as core counts continue to increase. To ensure that your memory is bound to the location where you want it to be, you may need to allocate some memory, bind it to the location that you want, “Pin” the memory so that it will never be paged out, and then touch each page in your memory allocation.