ISC 2017 Workshop Preview: Optimizing Linux Containers for HPC & Big Data Workloads

Christian Kniep is hosting a half-day Linux Container Workshop on Optimizing IT Infrastructure and High-Performance Workloads on June 23 in Frankfurt. “Docker as the dominant flavor of Linux Container continues to gain momentum within datacenter all over the world. It is able to benefit legacy infrastructure by leveraging the lower overhead compared to traditional, hypervisor-based virtualization. But there is more to Linux Containers – and Docker in particular, which this workshop will explore.”

New Site Compares Docker, Singularity, Shifter, and Univa Grid Engine Container Edition

A new site developed by Tin H compares the HPC virtualization capabilities of Docker, Singularity, Shifter, and Univa Grid Engine Container Edition. “They bring the benefits of container to the HPC world and some provide very similar features. The subtleties are in their implementation approach. MPI maybe the place with the biggest difference.”

Video: The State of Linux Containers

“With Docker v1.9 a new networking system was introduced, which allows multi-host network- ing to work out-of-the-box in any Docker environment. This talk provides an introduction on what Docker networking provides, followed by a demo that spins up a full SLURM cluster across multiple machines. The demo is based on QNIBTerminal, a Consul backed set of Docker Images to spin up a broad set of software stacks.”

Video: Docker, Monitoring, and SLURM Dashboards

“Based on a containerized HPC environment this talk shows of a state-of-the-art stack including performance monitoring, log event handling and GraphDB based inventory to provide insights into what is going on within a SLURM cluster. The framework used is QNIBTerminal incorporating the ELK stack, a graphite backend and neo4j as a GraphDB.”

Reproducibility of CAE-computations through Immutable Application Containers

Ensuring a reproducible result of HPC workloads no matter the platform used is key to settle the battle between operations (update fast, due to compliance / security) and end- user (never touch a running system). By pushing his docker related HPC research further, Christian is showcasing his results on Immutable Application Containers.

Will Containerization Eat Configuration Management?

Over at QNIB, Christian Kniep writes that his latest presentation examines intersection of Docker, Containerization, and Configuration Management. “In my humble opinion, Configuration Management might become a niche. As hard as it sounds.”

Slidecast: QNIBTerminal Plus InfiniBand – Containerized MPI Workloads

In this slidecast, Christian Kniep presents: QNIBTerminal Plus InfiniBand – Containerized MPI Workloads. “QNIB Solutions (early on called ‘QNIB Inc’) derives from the first project Christian did during his B.Sc. report, a InfiniBand monitoring suite. For the sake of the report is was named ‘OpenIBPM: Open Source InfiniBand Performance Monitoring’. Afterwards Christian renamed it to match his last name (Kniep): ‘QNIB: Qualified Networking with InfiniBand’. Since then QNIB becomes a pet project’s theme.”