One of the big surprises of the past few years has been the spectacular rise in the use of Python* in high-performance computing applications. With the latest releases of Intel® Distribution for Python, included in Intel® Parallel Studio XE 2019, the numerical and scientific computing capabilities of high-performance Python now extends to machine learning and data analytics.
But why Python? Because it’s easy to learn and comes with vast open source packages and libraries tailored for just about every computation domain, especially data analytics and machine learning.
Industrial strength data analytics involves some very serious math. A single application might employ many complex solutions requiring a significant effort to develop. With the Intel® Data Analytics Acceleration Library (Intel DAAL), the data scientist has all the building blocks they need, in Python, for all stages of data analysis from data acquisition through prediction and decision making. And, it scales from a single node to a large cluster with remote storage without additional effort.
At the center of all this, Intel DAAL is a highly optimized library of computationally intensive routines supporting best performance on Intel architectures including Intel Xeon® processors, Intel Core processors, Intel Atom processors, and Intel Xeon Phi™ processors. Intel DAAL provides Python with a rich set of algorithms, ranging from the most basic descriptive statistics for datasets to more advanced data mining and machine learning algorithms.
Python programmers can easily utilize Intel DAAL (daal4py) for developing robust, scalable, high performing data processing right out of the box, and immediately take advantage of its features and. Shown to give a substantial performance boost over alternatives, data scientists programming in Python with Intel DAAL can implement batch, online, clustering, and much more within their Python applications.
In many cases, existing Python applications will perform significantly better merely by switching to the Intel distribution. On distributed parallel systems, Intel Python supports the mpi4py library, which interfaces the Intel MPI Library over InfiniBand and the Intel Omni-Path communications fabric. The result is decreased latency and increased scaling for distributed Python applications.
Python packages, including Intel Math Kernel Library (Intel MKL), Intel Threading Building Blocks (Intel TBB), Intel Integrated Performance Primitives (Intel IPP), and Intel Data Analytics Acceleration Library (Intel DAAL), have all been optimized to take advantage of parallelism through the use of vectorization, multi-threading and multi-processing, as well as through the use of optimized communication across multiple nodes. Installation is simplified by providing packages in a binary form so that everything is preconfigured, and no compilation tools are needed, including all the dependences for running on popular OS platforms.
When you combine Python with the Numba* just-in-time (JIT) compiler, the Cython* compiler, and runtime packages built on Intel performance libraries such as Intel MKL and Intel DAAL, you now have C-like, near-native performance, plus Python bindings for your existing C and C++ libraries.
This means
- Faster machine learning with scikit-learn key algorithms accelerated with Intel DAAL
- TensorFlow* and Caffe* libraries optimized for Intel architecture (now available on the Intel channel at anaconda.org)
But there’s more: deep Python support with the Intel VTune™ Amplifier performance analyzer, part of the Intel Parallel Studio XE suite of tools. Intel VTune provides line-by-line source code profiling to help find and correct issues causing performance hotspots or bottlenecks in Python as well as Numba*, C and C++ source.
With many data scientists already utilizing Python for data analysis and machine learning, they experience generally poor performance in this application area. Intel Python Distribution includes an optimized scikit-learn machine learning package. Benchmarks show significant performance benefits with both data analysis and machine learning apps, especially on many-core platforms.
So, with Intel acceleration, it’s really no surprise to see the growing presence of Python in high-performance computing, big data, machine learning, and data science.
Get the Intel® Distribution for Python Now
Get your free 30-day trial of Intel® Parallel Studio XE today
