Sign up for our newsletter and get the latest HPC news and analysis.
Send me information from insideHPC:


Unify Your Analytics, and Keep Your Data Where It Suits You

In this sponsored post, Joy King, VP, Vertica Product Management & Product Marketing, believes that we need to stop our fixation with “data in one place.” The days of the single data repository are behind us. If you try doing that, you’ll incur so much data management time and cost that you’ll squander the savings even before you even get to the analysis stage. To put it simply, the goal is to unify, analyze, and act, because predictive analytics and proactive action is the definition of business success.

GigaOm Radar for Evaluating Data Warehouse Platforms

This new GigaOm Radar Report “GigaOm Radar for Evaluating Data Warehouse Platforms” provided by our friends over at Vertica, examines the leading platforms in the data warehouse marketplace, describes the fundamentals of the technology, identifies key criteria and evaluation metrics by which organizations can evaluate competing platforms, describes some potential technology developments to look out for in the future, and classifies platforms across those criteria and metrics.

GigaOm Radar for Evaluating Data Warehouse Platforms

This new GigaOm Radar Report provided by our friends over at Vertica, examines the leading platforms in the data warehouse marketplace, describes the fundamentals of the technology, identifies key criteria and  evaluation metrics by which organizations can evaluate competing platforms, describes some potential  technology developments to look out for in the future, and classifies platforms across those criteria and  metrics.

NetApp Deploys Iguazio’s Data Science Platform for Optimized Storage Management

Previously built on Hadoop, NetApp said it was also looking to modernize the service infrastructure “to reduce the complexities of deploying new AI services and the costs of running large-scale analytics. In addition, the shift was needed to enable real-time predictive AI, and to abstract deployment, allowing the technology to run on multi-cloud or on premises seamlessly.”

Designing HPC, Big Data, & Deep Learning Middleware for Exascale

DK Panda from Ohio State University presented this talk at the HPC Advisory Council Spain Conference. “This talk will focus on challenges in designing HPC, Big Data, and Deep Learning middleware for Exascale systems with millions of processors and accelerators. For the HPC domain, we will discuss about the challenges in designing runtime environments for MPI+X (PGAS OpenSHMEM/UPC/CAF/UPC++, OpenMP, and CUDA) programming models. Features and sample performance numbers from MVAPICH2 libraries will be presented.”

SC17 Invited Talk Preview: High Performance Machine Learning

Over at the SC17 Blog, Brian Ban begins his series of SC17 Session Previews with a look at a talk on High Performance Big Data. “Deep learning, using GPU clusters, is a clear example but many Machine Learning algorithms also need iteration, and HPC communication and optimizations.”

Podcast: PortHadoop Speeds Data Movement for Science

In this TACC Podcast, host Jorge Salazar interviews Xian-He Sun, Distinguished Professor of Computer Science at the Illinois Institute of Technology. Computer Scientists working in his group are bridging the file system gap with a cross-platform Hadoop reader called PortHadoop, short for portable Hadoop. “We tested our PortHadoop-R strategy on Chameleon. In fact, the speedup is 15 times faster,” said Xian-He Sun. “It’s quite amazing.”

Accelerating Hadoop, Spark, and Memcached with HPC Technologies

“This talk will present RDMA-based designs using OpenFabrics Verbs and heterogeneous storage architectures to accelerate multiple components of Hadoop (HDFS, MapReduce, RPC, and HBase), Spark and Memcached. An overview of the associated RDMA-enabled software libraries (being designed and publicly distributed as a part of the HiBD project for Apache Hadoop.”

Intel DAAL Accelerates Data Analytics and Machine Learning

Intel DAAL is a high-performance library specifically optimized for big data analysis on the latest Intel platforms, including Intel Xeon®, and Intel Xeon Phi™. It provides the algorithmic building blocks for all stages in data analysis in offline, batch, streaming, and distributed processing environments. It was designed for efficient use over all the popular data platforms and APIs in use today, including MPI, Hadoop, Spark, R, MATLAB, Python, C++, and Java.

Programming for High Performance Processors

“Managing the work on each node can be referred to as Domain parallelism. During the run of the application, the work assigned to each node can be generally isolated from other nodes. The node can work on its own and needs little communication with other nodes to perform the work. The tools that are needed for this are MPI for the developer, but can take advantage of frameworks such as Hadoop and Spark (for big data analytics). Managing the work for each core or thread will need one level down of control. This type of work will typically invoke a large number of independent tasks that must then share data between the tasks.”