A new site developed by Tin H compares the HPC virtualization capabilities of Docker, Singularity, Shifter, and Univa Grid Engine Container Edition.
Here is a comparison of HPCS Singularity vs NeRSC Shifter. They both bring the benefits of container to the HPC world. As such, both provide very similar features. The subtleties are in their implementation approach. MPI maybe the place with the biggest difference.
Let’s take a brief look at each of these platforms:
Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment.
In the high-performance computing (HPC) world, the focus is not necessarily on DevOps but on the ability to use containers to move applications from system to system,” writes Jeff Layton from AWS. “The desire is somewhat along the DevOps line, but the focus is really on mobility. Researchers or users can create applications and workflows using their environment of choice and then deploy the application or workflow on the production cluster for execution. The focus is not on the DevOps process but rather on application portability.”
Singularity is a container solution for researchers looking for “mobility of compute” so they can run their workflows anywhere. Developed by Gregory M. Kurtzer at LBNL, Singularity also allows you to leverage the resources of whatever host you are on. This includes HPC interconnects, resource managers, file systems, GPUs and/or accelerators, etc.
Containers have taken the tech world by storm, and while they may be a fantastic alternative to virtual machines (when the hardware abstraction is not necessary), they are not a great fit for application portability. Until now — Singularity combines software packaging models (such as RPM) with minimalistic containers to create very lightweight application bundles which can be simply executed and contained completely within their environment or be used to interact directly with the host file systems at native speeds. A Singularity application bundle can be as simple as containing a single binary application or as complicated as containing an entire workflow and is as flexible as you will need.
Shifter enables container images for HPC. In a nutshell, Shifter allows an HPC system to efficiently and safely allow end-users to run a docker image.
While Docker has taken the general computing world by storm in recent years, like container-based computing it has yet to be fully recognized in HPC. To facilitate the use of both of these tools in HPC, NERSC is enabling Docker-like container technology on its systems through a new, customized software package known as Shifter. Shifter, designed as a scalable method for deploying containers and user-defined images in an HPC environment, was developed at NERSC to improve flexibility and usability of its systems for increasingly data-intensive workloads.
Univa Grid Engine Container Edition features the unique ability to run containers at scale and blend containers with other workloads supporting heterogeneous applications and technology environments. Docker containers are being rapidly adopted by developer communities and provide a more responsive and efficient method to develop, modernize and release applications. IT operations managers want to incorporate containers within their production systems but require proven orchestration and scheduling software as well as configuration tools. Running Docker containers in a Grid Engine cluster fulfills these needs.
Recommended reading:
- A Container for HPC by Jeff Layton
- Docker for HPC in a Nutshell presentation by Andreas Schmidt
- Containerized MPI Workloads presentation by Christian Kniep
- UberCloud Application Containers by Wolfgang Gentzsch
- insideHPC Guide to Virtualization, Cloud and HPC
In all these valuable comparisons about strengths and weaknesses of containers and related technologies one fact should be taken into account: Docker has been developed (starting 6 years ago!) mainly with enterprise micro-services based applications in mind. Now when comparing Docker with Shifter and Singularity which both (recently) have been developed from the start off with HPC applications in mind it might be fair to include in such a comparison an HPC enhanced version of Docker which is able to handle MPI, Infiniband, GPU, remote HD viz, etc. One such version is the (HPC-enhanced) UberCloud Container based on Docker which is running on clouds since early 2015 like AWS, Azure, Atos, Advania, OzenCloud, CPU 24/7, Nephoscale, and others, and which hosts application software from ANSYS, CD-adapco/Siemens, OpenFOAM, Numeca, COMSOL, and more.