As an open source tool designed to navigate large amounts of data, Hadoop continues to find new uses in HPC. Managing a Hadoop cluster is different than managing an HPC cluster, however. It requires mastering some new concepts, but the hardware is basically the same and many Hadoop clusters now include GPUs to facilitate deep learning.
Video: Scalable High Performance Systems
September 8, 2015 by
In this video, Alexandru Iosup from the TU Delft presents: Scalable High Performance Systems. “During this masterclass, Alexandru discussed several steps towards addressing interesting new challenges which emerge in the operation of the datacenters that form the infrastructure of cloud services, and in supporting the dynamic workloads of demanding users. If we succeed, we may not only enable the advent of big science and engineering, and the almost complete automation of many large-scale processes, but also reduce the ecological footprint of datacenters and the entire ICT industry.”
Intel Invests in BlueData for Spinning Up Spark Clusters on the Fly
August 26, 2015 by
Today Intel Corporation and BlueData announced a broad strategic technology and business collaboration, as well as an additional equity investment in BlueData from Intel Capital. BlueData is a Silicon Valley startup that makes it easier for companies to install Big Data infrastructure, such as Apache Hadoop and Spark, in their own data centers or in the cloud.
Slidecast: IBM High Performance Services for Technical Computing in the Cloud
July 27, 2015 by
In this slidecast, Chris Porter and Jeff Kamiol from IBM describe how IBM High Performance Services deliver versatile, application-ready clusters in the cloud for organizations that need to quickly and economically add computing capacity for high performance application workloads.
Interview: How Univa Short Jobs Brings Low Latency to Financial Services
June 22, 2015 by
With the launch of Univa Small Jobs add-on for Univa Grid Engine, the company, the company offers “the world’s most efficient processing and lowest latency available for important tasks like real-time trading, transactions, and other critical applications.” To learn more, we caught up with Univa President & CEO Gary Tyreman.
Bright Cluster Manager Speeds Deployment of Apache Hadoop
June 9, 2015 by
Today Bright Computing announced a significant update to distribution-agnostic Bright Cluster Manager for Apache Hadoop at the Hadoop Summit North America 2015.
Video: Understanding Hadoop Performance on Lustre
April 27, 2015 by
“In this talk, Seagate presents details on its efforts and achievements around improving Hadoop performance on Lustre including a summary on why and how HDFS and Lustre are different and how those differences affect Hadoop performance on Lustre compared to HDFS, Hadoop ecosystem benchmarks and best practices on HDFS and Lustre, Seagate’s open-source efforts to enhance performance of Lustre within “diskless” compute nodes involving core Hadoop source code modification (and the unexpected results), and general takeaways ways on running Hadoop on Lustre more rapidly.”
Deploying Hadoop on Lustre Storage: Lessons Learned and Best Practices
April 24, 2015 by
In this video from LUG 2015 in Denver, J.Mario Gallegos from Dell presents: Deploying Hadoop on Lustre Storage: Lessons Learned and Best Practices. “Merging of strengths of both technologies to solve big data problems permits harvesting the power of HPC clusters on very fast storage.”
Penguin Computing Launches Scyld ClusterWare for Hadoop
February 25, 2015 by
Today Penguin Computing announced Scyld ClusterWare for Hadoop, adding greater capability to the company’s existing Scyld ClusterWare high performance computing cluster management solution.
Video: SGI UV Finds the Needle in the Big Data Haystack
February 19, 2015 by
According to IDC, SGI has shipped approximately 8 percent of of all the Hadoop servers in production today. In fact, did you know that SGI introduced the word “Big Data” to supercomputing in 1996? Jorge Titinger, SGI President and CEO, shares SGI’s history in helping to design, develop, and deploy Hadoop clusters. (NOTE: Straw was substituted for actual hay to avoid any potential allergic reactions.)
