Adam Huffman
In this video from FOSDEM’17, Adam Huffman from the Francis Crick Institute presents: The Marriage of Cloud, HPC and Containers.
“We will present experiences of supporting HPC/HTC workloads on private cloud resources, with ideas for how to do this better and description of trends for non-traditionalHPC resource provision. I will discuss my work as part of the Operations Team for the eMedLab private cloud, which is a large-scale (6000-core, 5PB)biomedical research cloud using HPC hardware, aiming to support HPC workloads.
This includes an account of the technical challenges of marrying traditional HPC components such as parallel file systems with the rather different expectations of the OpenStack cloud management framework. Other challenges faced include educating users about the basics of virtual environments, providing access to restricted-license large-scale (800TB) datasets in a multi-tenant environment, and achieving good performance inside virtual HPC clusters. Looking to the future, container orchestration engines such as Kubernetes provide a very promising approach, as the containerization of scientific applications gains momentum, and I will describe existing and future work in this area.”
Adam Huffman is responsible for providing research computing infrastructure in the BDI that matches the ambitious goals of the institute and its researchers. He is building a team in the BDI and working in conjunction with the ResComp team in WCHG. He has been working in scientific computing since 1999, including at the University of Manchester, Imperial College London and the Francis Crick Institute.
