Anderson Banihirwe from NCAR gave this talk at SciPy 2019. “This talk demonstrates how to use Dask and Jupyter on large high-performance computing (HPC) systems to scale and accelerate large interactive data analysis tasks — effectively turning HPC systems into interactive big-data platforms. We will introduce dask-jobqueue which allows users to seamlessly deploy and scale dask on HPC clusters that use a variety of job queuing systems such as PBS, Slurm, SGE, and LSF. We will also introduce dask-mpi, a Python package that makes deploying Dask easy from within a distributed MPI environment.”
SDSC Awarded NSF Grant for Triton Stratus
July 30, 2019 by
The National Science Foundation has awarded SDSC a two-year grant worth almost $400,000 to deploy a new system called Triton Stratus. “Triton Stratus will provide researchers with improved facilities for utilizing emerging computing paradigms and tools, namely interactive and portal-based computing, and scaling them to commercial cloud computing resources. Researchers, especially data scientists, are increasingly using toolssuch as Jupyter notebooks and RStudio to implement computational and data analysis functions and workflows.”
RCE-Podcast Looks at Project Jupyter for Interactive Data Science
November 2, 2017 by
In this RCE Podcast, Brock Palen and Jeff Squyres discuss Jupyter with Dr. Brian Granger from Cal Poly State University. “Jupyter is a non-profit, open-source project, born out of the IPython Project in 2014 as it evolved to support interactive data science and scientific computing across all programming languages.”
FlyElephant 2.0 Improves HPC Collaboration Features
July 27, 2016 by
Today the FlyElephant team announced the release of the FlyElephant 2.0 platform for High Performance Computing. Versioin 2.0 enhancements include: internal expert community, collaboration on projects, public tasks, Docker and Jupyter support, a new file storage system and work with HPC clusters.
Video: A Computational Future for Science Education
July 10, 2016 by
“This talk will describe one new effort to embed best practices for reproducible scientific computing into traditional university curriculum. In particular, a set of open source, liberally licensed, IPython (now Jupyter) notebooks are being developed and tested to accompany a book “Effective Computation in Physics.” These interactive lecture materials lay out in-class exercises for a project-driven upper-level undergraduate course and are accordingly intended to be forked, modified and reused by professors across universities and disciplines.”
