In this TACC podcast, Joe Stubbs from the Texas Advanced Computing Centter describes potential benefits to scientists of open container platform Docker in supporting reproducibility, NSF-funded Agave API. “As more scientists share not only their results but their data and code, Docker is helping them reproduce the computational analysis behind the results. What’s more, Docker is one of the main tools used in the Agave API platform, a platform-as-a-service solution for hybrid cloud computing developed at TACC and funded in part by the National Science Foundation.”
“Research computational workflows consist of several pieces of third party software and, because of their experimental nature, frequent changes and updates are commonly necessary thus raising serious deployment and reproducibility issues. Docker containers are emerging as a possible solution for many of these problems, as they allow the packaging of pipelines in an isolated and self-contained manner. This presentation will introduce our experience deploying genomic pipelines with Docker containers at the Center for Genomic Regulation (CRG). I will discuss how we implemented it, the main issues we faced, the pros and cons of using Docker in an HPC environment including a benchmark of the impact of containers technology on the performance of the executed applications.”
In this special guest feature from Scientific Computing World, Dr Bruno Silva from The Francis Crick Institute in London writes that new cloud technologies will make the cloud even more important to scientific computing. “The emergence of public cloud and the ability to cloud-burst is actually the real game-changer. Because of its ‘infinite’ amount of resources (effectively always under-utilized), it allows for a clear decoupling of time-to-science from efficiency. One can be somewhat less efficient in a controlled fashion (higher cost, slightly more waste) to minimize time-to-science when required (in burst, so to speak) by effectively growing the computing estate available beyond the fixed footprint of local infrastructure – this is often referred to as the hybrid cloud model. You get both the benefit of efficient infrastructure use, and the ability to go beyond that when strictly required.”
“Containers wrap up software with all its dependencies in packages that can be executed anywhere. This can be specially useful in HPC environments where, often, getting the right combination of software tools to build applications is a daunting task. However, typical container solutions such as Docker are not a perfect fit for HPC environments. Instead, Shifter is a better fit as it has been built from the ground up with HPC in mind. In this talk, we show you what Shifter is and how to leverage from the current Docker environment to run your ap- plications with Shifter.”
“With Docker v1.9 a new networking system was introduced, which allows multi-host network- ing to work out-of-the-box in any Docker environment. This talk provides an introduction on what Docker networking provides, followed by a demo that spins up a full SLURM cluster across multiple machines. The demo is based on QNIBTerminal, a Consul backed set of Docker Images to spin up a broad set of software stacks.”
Over at the UberCloud, Wolfgang Gentzsch writes that, despite the ever increasing complexity of CAE tools, hardware, and system components engineers have never been this close to ubiquitous CAE as a common tool for every engineer.
On December 18, Oracle closed an agreement to acquire StackEngine, an Austin-based startup focused on Docker automation. All StackEngine employees will be joining Oracle as part of Oracle Public Cloud.
“We have enabled virtualization for HPC but it’s important to bring the benefits of virtualization to end researchers in a way they can use it, right? So what we have done is we have created the solution plus VMware High-Performance Analytics, which allows researchers to author their own workloads, they can collaborate it, they can clone it, then they can share it with other researchers. And they can modify their workload – they can fine tune it.”
Today Cray announced plans to deliver an HPC-optimized Docker solution for the Cray XC line of supercomputers. Available in 2016, the HPC virtualization capabilities will be offered for the Cray CS400, Cray XE and Cray XK platforms.
Today NUMECA announced that its complete suite of CFD software powered by UberCloud application software containers are now available as a service in any cloud.
Until recently it was my firm belief that cloud computing for engineering applications is one of the next big challenges we have to solve,” said Professor Charles Hirsch, President and founder of NUMECA International, and world-renowned expert on Computational Fluid Dynamics. “But when we came across UberCloud’s new application container technology and containerized all our CFD software packages we were surprised about the ease of use and access to any computing system on demand.”