Sign up for our newsletter and get the latest HPC news and analysis.
Send me information from insideHPC:


Charliecloud: Unprivileged Containers for User-Defined Software Stacks

“What if I told you there was a way to allow your customers and colleagues to run their HPC jobs inside the Docker containers they’re already creating? or an easily learned, easily employed method for consistently reproducing a particular application environment across numerous Linux distributions and platforms? There is. In this talk/tutorial session, we’ll explore the problem domain and all the previous solutions, and then we’ll discuss and demo Charliecloud, a simple, streamlined container runtime that fills the gap between Docker and HPC — without requiring HPC Admins to lift a finger!”

Introducing the SPEC High Performance Group and HPC Benchmark Suites

Robert Henschel from Indiana University gave this talk at the Swiss HPC Conference. “In this talk, I will present an overview of the High Performance Group as well as SPEC’s benchmarking philosophy in general. Most everyone knows SPEC for the SPEC CPU benchmarks that are heavily used when comparing processor performance, but the High Performance Group specifically focusses on whole system benchmarking utilizing the parallelization paradigms common in HPC, like MPI, OpenMP and OpenACC.”

Ceph on the Brain: Storage and Data-Movement Supporting the Human Brain Project

Adrian Tate from Cray and Stig Telfer from StackHPC gave this talk at the 2018 Swiss HPC Conference. “This talk will describe how Cray, StackHPC and the HBP co-designed a next-generation storage system based on Ceph, exploiting complex memory hierarchies and enabling next-generation mixed workload execution. We will describe the challenges, show performance data and detail the ways that a similar storage setup may be used in HPC systems of the future.”

Industry Insights: Download the Results of our AI & HPC Perceptions Survey

The results from our HPC & AI peception survey are here. “90 percent of all respondents felt that their business will ultimately be impacted by AI. Although almost all respondents see AI as playing a role in the future of the business, the survey also revealed the top three industries that will see the most impact. Healthcare came in first, followed by life sciences, and finance/transportation tied in third place. The possibilities of AI are seemingly endless. And the shift has already begun.”

Scratch to Supercomputers: Bottoms-up Build of Large-scale Computational Lensing Software

Gilles Fourestey from EPFL gave this talk at the Swiss HPC Conference. “LENSTOOL is a gravitational lensing software that models mass distribution of galaxies and clusters. It is used to obtain sub-percent precision measurements of the total mass in galaxy clusters and constrain the dark matter self-interaction cross-section, a crucial ingredient to understanding its nature.”

Shifter – Docker Containers for HPC

Alberto Madonaa gave this talk at the Swiss HPC Conference. “In this work we present an extension to the container runtime of Shifter that provides containerized applications with a mechanism to access GPU accelerators and specialized networking from the host system, effectively enabling performance portability of containers across HPC resources. The presented extension makes possible to rapidly deploy high-performance software on supercomputers from containerized applications that have been developed, built, and tested in non-HPC commodity hardware, e.g. the laptop or workstation of a researcher.”

Accelerating Ceph with RDMA and NVMe-oF

Haodong Tang from Intel gave this talk at the 2018 Open Fabrics Workshop. “Efficient network messenger is critical for today’s scale-out storage systems. Ceph is one of the most popular distributed storage system providing a scalable and reliable object, block and file storage services. As the explosive growth of Big Data continues, there’re strong demands leveraging Ceph build high performance & ultra-low latency storage solution in the cloud and bigdata environment. The traditional TCP/IP cannot satisfy this requirement, but Remote Direct Memory Access (RDMA) can.”

Amazon and Libfabric: A case study in flexible HPC Infrastructure

Brian Barrett from Amazon gave this talk at the 2018 OpenFabrics Workshop. “As network performance becomes a larger bottleneck in application performance, AWS is investing in improving HPC network performance. Our initial investment focused on improving performance in open source MPI implementations, with positive results. Recently, however, we have pivoted to focusing on using libfabric to improve point to point performance.”

Job of the Week: Director of Computing at ECMWF

“Supercomputing capability plays a key role in ECMWF’s success and in its ability to implement its strategic vision to 2025. In order to meet the required flexibility for future growth, ECMWF’s data centre is being relocated to Bologna, Italy and will be operational in 2020. The new Director of Computing will be responsible for delivering this challenging transition, fit-out and transformation, whilst ensuring that ECMWF’s computing capability continues to support some of the most critical scientific advances of our time.”

Exploiting HPC Technologies for Accelerating Big Data Processing and Associated Deep Learning

DK Panda from Ohio State University gave this talk at the Swiss HPC Conference. “This talk will provide an overview of challenges in accelerating Hadoop, Spark, and Memcached on modern HPC clusters. An overview of RDMA-based designs for Hadoop (HDFS, MapReduce, RPC and HBase), Spark, Memcached, Swift, and Kafka using native RDMA support for InfiniBand and RoCE will be presented. Enhanced designs for these components to exploit NVM-based in-memory technology and parallel file systems (such as Lustre) will also be presented.”