Sign up for our newsletter and get the latest HPC news and analysis.
Send me information from insideHPC:


Supercomputing by API: Connecting Modern Web Apps to HPC

In this video from OpenStack Australia, David Perry from the University of Melbourne presents: Supercomputing by API – Connecting Modern Web Apps to HPC. “OpenStack is a free and open-source set of software tools for building and managing cloud computing platforms for public and private clouds. OpenStack Australia Day is the region’s largest, and Australia’s best, conference focusing on Open Source cloud technology. Gathering users, vendors and solution providers, OpenStack Australia Day is an industry event to showcase the latest technologies and share real-world experiences of the next wave of IT virtualization.”

RCE Podcast Looks at Shifter Containers for HPC

In this RCE Podcast, Brock Palen and Jeff Squyres speak with Shane Canon and Doug Jacobsen from NERSC, the authors of Shifter. “Shifter is a prototype implementation that NERSC is developing and experimenting with as a scalable way of deploying containers in an HPC environment. It works by converting user or staff generated images in Docker, Virtual Machines, or CHOS (another method for delivering flexible environments) to a common format.”

How Charliecloud Simplifies Big Data Supercomputing at LANL

“Los Alamos has lots of supercomputing power, and we do lots of simulations that are well supported here. But we’ve found that Big Data analysis projects need to use different frameworks, which often have dependencies that differ from what we have already on the supercomputer. So, we’ve developed a lightweight ‘container’ approach that lets users package their own user defined software stack in isolation from the host operating system.”

Learning from ZFS to Scale Storage on and under Containers

“What is so new about the container environment that a new class of storage software is emerging to address these use cases? And can container orchestration systems themselves be part of the solution? As is often the case in storage, metadata matters here. We are implementing in the open source OpenEBS.io some approaches that are in some regards inspired by ZFS to enable much more efficient scale out block storage for containers that itself is containerized. The goal is to enable storage to be treated in many regards as just another application while, of course, also providing storage services to stateful applications in the environment.”

ISC 2017 Workshop Preview: Optimizing Linux Containers for HPC & Big Data Workloads

Christian Kniep is hosting a half-day Linux Container Workshop on Optimizing IT Infrastructure and High-Performance Workloads on June 23 in Frankfurt. “Docker as the dominant flavor of Linux Container continues to gain momentum within datacenter all over the world. It is able to benefit legacy infrastructure by leveraging the lower overhead compared to traditional, hypervisor-based virtualization. But there is more to Linux Containers – and Docker in particular, which this workshop will explore.”

Penguin Computing Adds Support for Singularity Containers on POD HPC Cloud

Today Penguin Computing announced support for Singularity containers on its Penguin Computing On-Demand (POD) HPC Cloud and Scyld ClusterWare HPC management software. “Our researchers are excited about using Singularity on POD,” said Jon McNally, Chief HPC Architect at ASU Research Computing. “Portability and the ability to reproduce an environment is key to peer reviewed research. Unlike other container technologies, Singularity allows them to run at speed and scale.”

Greg Kurtzer of LBNL Launches SingularityWare, LLC

Over at the Singularity Blog, Greg Kurtzer writes that he has created a new organization, SingularityWare, LLC. In partnership with RStor, the new company will be dedicated to further developing Singularity, supporting the associated open source community and growing the project. “In addition to continuing my leadership of Singularity (and the new LLC), I will be maintaining my association with Lawrence Berkeley National Laboratory, as a scientific advisor as well as continuing other efforts I am associated with (e.g. Warewulf and OpenHPC).”

HPC Workflows Using Containers

“In this talk we will discuss a workflow for building and testing Docker containers and their deployment on an HPC system using Shifter. Docker is widely used by developers as a powerful tool for standardizing the packaging of applications across multiple environments, which greatly eases the porting efforts. On the other hand, Shifter provides a container runtime that has been specifically built to fit the needs of HPC. We will briefly introduce these tools while discussing the advantages of using these technologies to fulfill the needs of specific workflows for HPC, e.g., security, high-performance, portability and parallel scalability.”

Video: InfiniBand Virtualization

“Infiniband Virtualization allows a single Channel Adapter to present multiple transport endpoints that share the same physical port. To software, these endpoints are exposed as independent Virtual HCAs (VHCAs), and thus may be assigned to different software entities, such as VMs. VHCAs are visible to Subnet Management, and are managed just like physical HCAs. We will cover the Virtualization model, management, addressing modes, and discuss deployment considerations.”

Video: Containerizing Distributed Pipes

Hagen Toennies from Gaikai Inc. presented these Best Practices at the 2017 HPC Advisory Council Stanford Conference. “In this talk we will present how we enable distributed, Unix style programming using Docker and Apache Kafka. We will show how we can take the famous Unix Pipe Pattern and apply it to a Distributed Computing System.”