Sign up for our newsletter and get the latest HPC news and analysis.
Send me information from insideHPC:


Intel Performance Libraries Accelerate Python Performance for HPC and Data Science

Python is now the most popular programming language, according to IEEE Spectrum’s fifth annual interactive ranking of programming languages, ahead of C++ and C. Recent Intel Distributions for Python show that real HPC performance can be achieved with compilers and library packages optimized for the latest Intel architectures. Moreover, the library packages targeted for big data analysis and numerical computation included in this distribution now support scaling for multi-core and many-core processors as well as distributed cluster and cloud infrastructures.

Machine Learning with Python: Distributed Training and Data Resources on Blue Waters

Aaron Saxton from NCSA gave this talk at the Blue Waters Symposium. “Blue Waters currently supports TensorFlow 1.3, PyTorch 0.3.0 and we hope to support CNTK and Horovod in the near future. This tutorial will go over the minimum ingredients needed to do distributed training on Blue Waters with these packages. What’s more, we also maintain an ImageNet data set to help researchers get started training CNN models. I will review the process by which a user can get access to this data set.”

Python Can Do It

“Python remains a single threaded environment with the global interpreter lock as the main bottleneck. Threads must wait for other threads to complete before starting to do their assigned work. The result of this model is that production code is produced that is too slow to be useful for large simulations.”

Performance Insights Using the Intel Advisor Python API

Tuning a complex application for today’s heterogeneous platforms requires an understanding of the application itself as well as familiarity with tools that are available for assisting with analyzing where in the code itself to look for bottlenecks.  The process for optimizing the performance of an application, in general, requires the following steps that are most likely applicable for a wide range of applications.

HPC Carpentry Learning Portal Offers an Intro to HPC

The good folks at HPC Carpentry have posted a new set of teaching materials designed to help new users take advantage of high-performance computing systems. No prior computational experience is required – these lessons are ideal for either an in-person workshop or independent study. “HPC Carpentry is not an organization – it is merely a set of publicly available teaching materials designed to make the task of teaching HPC a little easier. We welcome all contributions, in particular adaptations of our Intro to HPC lesson for other schedulers besides SLURM.”

Intel MPI Library 2017 Focuses on Intel Multi-core/Many-Core Clusters

With the release of Intel Parallel Studio XE 2017, the focus is on making applications perform better on Intel architecture-based clusters. Intel MPI Library 2017, a fully integrated component of Intel Parallel Studio XE 2017, implements the high-performance MPI-3.1 specification on multiple fabrics. It enables programmers to quickly deliver the best parallel performance, even if you change or upgrade to new interconnects, without requiring changes to the software or operating environment.

Video: Scientific Visualization with Python

In this video, Dr. Marcelo Ponce from SciNet presents: Scientific Visualization with Python. “SciNet is Canada’s largest supercomputer centre, providing Canadian researchers with computational resources and expertise necessary to perform their research on scales not previously possible in Canada. We help power work from the biomedical sciences and aerospace engineering to astrophysics and climate science.”

Intel Releases Optimized Python for HPC

“By implementing popular Python packages such as NumPy, SciPy, scikit-learn, to call the Intel Math Kernel Library (Intel MKL) and the Intel Data Analytics Acceleration Library (Intel DAAL), Python applications are automatically optimized to take advantage of the latest architectures. These libraries have also been optimized for multithreading through calls to the Intel Threading Building Blocks (Intel TBB) library. This means that existing Python applications will perform significantly better merely by switching to the Intel distribution.”

Podcast: How PyLadies are Increasing Diversity in Coding and Data Science

In this Intel Chip Chat podcast, Dr. Julie Krugler Hollek, co-organizer of PyLadies San Francisco and Data Scientist at Twitter, joins Allyson Klein to discuss efforts to democratize participation in open source communities and the future of data science. “PyLadies helps people who identify as women become participants in open source Python projects like The SciPy Stack, a specification that provides access to machine learning and data visualization tools.”

Python and HPC

“In the HPC domain, Python can be used to develop a wide range of applications. While tight loops may still need to be coded in C or FORTRAN, Python can still be used. As more systems become available with coprocessors or accelerators, Python can be used to offload the main CPU and take advantage of the coprocessor. pyMIC is a Python Offload Module for the Intel Xeon Phi Coprocessor and is available at popular open source code repositories.”