Sign up for our newsletter and get the latest HPC news and analysis.
Send me information from insideHPC:

Charliecloud: Unprivileged Containers for User-Defined Software Stacks

“What if I told you there was a way to allow your customers and colleagues to run their HPC jobs inside the Docker containers they’re already creating? or an easily learned, easily employed method for consistently reproducing a particular application environment across numerous Linux distributions and platforms? There is. In this talk/tutorial session, we’ll explore the problem domain and all the previous solutions, and then we’ll discuss and demo Charliecloud, a simple, streamlined container runtime that fills the gap between Docker and HPC — without requiring HPC Admins to lift a finger!”

Video: Addressing Key Science Challenges with Adversarial Neural Networks

Wahid Bhimji from NERSC gave this talk at the 2018 HPC User Forum in Tucson. “Machine Learning and Deep Learning are increasingly used to analyze scientific data, in fields as diverse as neuroscience, climate science and particle physics. In this page you will find links to examples of scientific use cases using deep learning at NERSC, information about what deep learning packages are available at NERSC, and details of how to scale up your deep learning code on Cori to take advantage of the compute power available from Cori’s KNL nodes.”

Introducing the SPEC High Performance Group and HPC Benchmark Suites

Robert Henschel from Indiana University gave this talk at the Swiss HPC Conference. “In this talk, I will present an overview of the High Performance Group as well as SPEC’s benchmarking philosophy in general. Most everyone knows SPEC for the SPEC CPU benchmarks that are heavily used when comparing processor performance, but the High Performance Group specifically focusses on whole system benchmarking utilizing the parallelization paradigms common in HPC, like MPI, OpenMP and OpenACC.”

Ceph on the Brain: Storage and Data-Movement Supporting the Human Brain Project

Adrian Tate from Cray and Stig Telfer from StackHPC gave this talk at the 2018 Swiss HPC Conference. “This talk will describe how Cray, StackHPC and the HBP co-designed a next-generation storage system based on Ceph, exploiting complex memory hierarchies and enabling next-generation mixed workload execution. We will describe the challenges, show performance data and detail the ways that a similar storage setup may be used in HPC systems of the future.”

Pawsey Supercomputing Centre Hosts GPU Hackathon this week

Australia’s Pawsey Supercomputing Centre is hosting a GPU Hackathon this week in Perth, Australia. “The GPU Hackathon is a free event taking place at Esplanade Hotel in Fremantle, from Monday 16 April to Friday 20 April. Six teams from Australia, the United States, and Europe, are gathering in Perth for this 5-day event to adapt their applications for GPU architectures.”

Scratch to Supercomputers: Bottoms-up Build of Large-scale Computational Lensing Software

Gilles Fourestey from EPFL gave this talk at the Swiss HPC Conference. “LENSTOOL is a gravitational lensing software that models mass distribution of galaxies and clusters. It is used to obtain sub-percent precision measurements of the total mass in galaxy clusters and constrain the dark matter self-interaction cross-section, a crucial ingredient to understanding its nature.”

Shifter – Docker Containers for HPC

Alberto Madonaa gave this talk at the Swiss HPC Conference. “In this work we present an extension to the container runtime of Shifter that provides containerized applications with a mechanism to access GPU accelerators and specialized networking from the host system, effectively enabling performance portability of containers across HPC resources. The presented extension makes possible to rapidly deploy high-performance software on supercomputers from containerized applications that have been developed, built, and tested in non-HPC commodity hardware, e.g. the laptop or workstation of a researcher.”

Amazon and Libfabric: A case study in flexible HPC Infrastructure

Brian Barrett from Amazon gave this talk at the 2018 OpenFabrics Workshop. “As network performance becomes a larger bottleneck in application performance, AWS is investing in improving HPC network performance. Our initial investment focused on improving performance in open source MPI implementations, with positive results. Recently, however, we have pivoted to focusing on using libfabric to improve point to point performance.”

Exploiting HPC Technologies for Accelerating Big Data Processing and Associated Deep Learning

DK Panda from Ohio State University gave this talk at the Swiss HPC Conference. “This talk will provide an overview of challenges in accelerating Hadoop, Spark, and Memcached on modern HPC clusters. An overview of RDMA-based designs for Hadoop (HDFS, MapReduce, RPC and HBase), Spark, Memcached, Swift, and Kafka using native RDMA support for InfiniBand and RoCE will be presented. Enhanced designs for these components to exploit NVM-based in-memory technology and parallel file systems (such as Lustre) will also be presented.”

New Types of Memory, their support in Linux, and how to use them via RDMA

Christoph Lameter from Jump Trading LLC gave this talk at the OpenFabrics Workshop. “Recently new types of memory have shown up like HBM (High Bandwidth Memory), Optane, 3DXpoint, NVDIMM, NVME and various “nonvolatile” types memory. This talk gives a brief rundown on what is available and gives some example on how the vendors enable the actual use of this memory in the operating system (f.e. DAX and filesystems) and then show how an application would make use of this memory. In particular then we will be looking at what considerations are important for the use of RDMA to those memory devices.”