A new site developed by Tin H compares the HPC virtualization capabilities of Docker, Singularity, Shifter, and Univa Grid Engine Container Edition. “They bring the benefits of container to the HPC world and some provide very similar features. The subtleties are in their implementation approach. MPI maybe the place with the biggest difference.”
“With Docker v1.9 a new networking system was introduced, which allows multi-host network- ing to work out-of-the-box in any Docker environment. This talk provides an introduction on what Docker networking provides, followed by a demo that spins up a full SLURM cluster across multiple machines. The demo is based on QNIBTerminal, a Consul backed set of Docker Images to spin up a broad set of software stacks.”
“Based on a containerized HPC environment this talk shows of a state-of-the-art stack including performance monitoring, log event handling and GraphDB based inventory to provide insights into what is going on within a SLURM cluster. The framework used is QNIBTerminal incorporating the ELK stack, a graphite backend and neo4j as a GraphDB.”
Ensuring a reproducible result of HPC workloads no matter the platform used is key to settle the battle between operations (update fast, due to compliance / security) and end- user (never touch a running system). By pushing his docker related HPC research further, Christian is showcasing his results on Immutable Application Containers.
Over at QNIB, Christian Kniep writes that his latest presentation examines intersection of Docker, Containerization, and Configuration Management. “In my humble opinion, Configuration Management might become a niche. As hard as it sounds.”
In this slidecast, Christian Kniep presents: QNIBTerminal Plus InfiniBand – Containerized MPI Workloads. “QNIB Solutions (early on called ‘QNIB Inc’) derives from the first project Christian did during his B.Sc. report, a InfiniBand monitoring suite. For the sake of the report is was named ‘OpenIBPM: Open Source InfiniBand Performance Monitoring’. Afterwards Christian renamed it to match his last name (Kniep): ‘QNIB: Qualified Networking with InfiniBand’. Since then QNIB becomes a pet project’s theme.”