This sponsored post covers how Intel Performance Libraries are working to ramp up python performance.
Surprise! Python* is now the most popular programming language, according to IEEE Spectrum’s fifth annual interactive ranking of programming languages. C++ and C followed closely behind in this year’s snapshot, which is based on data gathered in June 2018.
But, Python? Really?
Well, the fact is that Python has successfully found its way into nearly all the major technical application areas previously ruled by C, C++, and Fortran. These include data science and machine learning, financial transactions, academic research, and, most importantly, numerical computing for science and engineering.
Why is this surprising? Because Python, which was introduced in 1991 as a good way to learn to program, has long been thought to be a slow interpretive language that’s just not the right choice for large industrial-strength applications.
On the other hand, Python is easy to learn and understand and has a very active user community. What attracts programmers to Python is its clean style and the vast number of open source library packages that seem to have solutions for everything worth doing. If your day job is data analysis or computational physics, and programming is just one of the tools you use to get work done, Python has become the preferred choice. And, it’s free and open source.
But, using Python for HPC?
Yes, but you need to use the Intel® Distribution for Python*, part of Intel Parallel Studio XE . Because when you combine Python with the Numba just-in-time (JIT) compiler, the Cython compiler, and runtime packages built on Intel performance libraries such as Intel Math Kernel Library (Intel MKL) and Intel Data Analytics Acceleration Library (Intel DAAL), you now have C-like, near-native performance, plus Python bindings for your existing C and C++ libraries.
Recent Intel Distributions for Python show that real HPC performance can be achieved with compilers and library packages optimized for the latest Intel architectures. Moreover, the library packages targeted for big data analysis and numerical computation included in this distribution now support scaling for multi-core and many-core processors as well as distributed cluster and cloud infrastructures.
By implementing popular Python packages such as NumPy, SciPy, scikit-learn*, to call the Intel Math Kernel Library and the Intel Data Analytics Acceleration Library (Intel DAAL), Python applications are automatically optimized to take advantage of the latest Intel® Core™ and Intel® Xeon® processors.
For distributed parallel systems, Intel Python supports the mpi4py library, which interfaces the Intel MPI Library over InfiniBand and the Intel Omni-Path communications fabric. The result is decreased latency and increased scaling for distributed Python applications.
The Intel Distribution for Python takes advantage of the Intel® Advanced Vector Extensions (Intel® AVX) and multiple cores in the latest Intel architectures. By utilizing the highly optimized Intel MKL BLAS and LAPACK routines, key functions run up to 200 times faster on servers and 10 times faster on desktop systems.
This means that existing Python applications will perform significantly better merely by switching to the Intel distribution.
But it’s not just the optimized libraries and compilers that generate vector/parallel code for the latest Intel processors; there is first-class Python support with the Intel VTune™ Amplifier performance analyzer, part of the Intel Parallel Studio XE suite of tools. Intel Vtune provides line-by-line source code profiling to help find and correct issues causing performance hotspots or bottlenecks in Python as well as C and C++ source.
Many data scientists already utilize Python for data analysis. To address the generally poor performance seen in this application area, Intel includes an optimized scikit-learn machine learning package. Benchmarks show significant performance benefits with both, especially on many-core platforms.
So, with Intel acceleration, it’s really no surprise to see the growing presence of Python in high-performance computing, big data, machine learning, and data science.
Get the Intel® Distribution for Python Now
Get your free 30-day trial of Intel® Parallel Studio XE today
