Sign up for our newsletter and get the latest HPC news and analysis.
Send me information from insideHPC:

Optimizing Your Code for Big Data

devtoolsSponsored Post

Big data applications have the potential to change many aspects of how large systems can be used. With the massive amounts of data being produced hourly and daily, powerful systems are needed to help data scientists understand the data and make sense of it. If a computer system is not able to help the user make informed decisions due to inefficient use of the resources, then the value of the purchase and time to implement the service is decreased.

As a developer trying to create an easy to use and powerful system that must deal with a wide variety of data, any tool that speeds up both development and performance will be very valuable. Rather than create a new analytics program from scratch, there are tools available that can speed up both the development time as well as the execution time. Real time decision making can help organizations to increase profits or gain new insights into simulations. Fast decisions require optimized software and speedy hardware.

Libraries that are tuned to the underlying hardware architecture can increase performance tremendously. Higher level libraries such at the Intel Data Analytics Acceleration Library (Intel DAAL) can assist the developer with highly tuned algorithms for data analysis as well as machine learning. Intel DAAL functions can be called within other, more comprehensive frameworks that deal with the various types of data and storage, increasing the performance and lowering the development time of a wide range of applications.

At a lower level that delivers maximum performance, the Intel Math Kernel Library (Intel MKL) can accelerate math functions by supplying highly optimized routines. Machine learning applications can be developed faster with Intel MKL, as neural network primitives are included as well.

These functions are optimized to take advantage of the underlying hardware architecture, and are tuned and threaded for both CPUs and the Intel Xeon Phi processors.

It is important to note that these libraries can be called from applications developed in Python, Java or C++, all favorites of those developing Big Data and analytics type applications. For applications that are in the early phases of specifications or existing applications that need a performance boost, using the Intel DAAL and Intel MKL products will definitely show their benefits quickly. Make sure to download these totally free products today.

Transform Data into Opportunity Accelerate analysis: Intel® Data Analytics Acceleration Library

Resource Links: