insideHPC » HPC Software

OpenFabrics Alliance to Host RDMA Training Courses in New Hampshire

Rich Brueckner — Wed, 22 May 2013 16:25:52 +0000

Today the OpenFabrics Alliance (OFA) announced it will host two training classes at the University of New Hampshire Interoperability Lab. The two-day courses, instructed by Dr. Robert D. Russell and Rupert Dance, will provide attendees with hands-on training for managing InfiniBand Fabric in addition to coding applications directly to the VERBS API using OpenFabrics Enterprise Distribution (OFED) and related software tools.

Writing Application Programs for RDMA Using OFA Software
- Date: June 3-4, 2013 from 8 a.m. – 6 p.m.
- Who Should Attend: Application developers skilled in C programming and familiar with sockets, but with little or no experience programming with OpenFabrics Software.
- Register now.

InfiniBand Fabric Administration
- Date: June 5-6, 2013 from 8 a.m. – 6 p.m.
- Who Should Attend: System Administrators, System Architects, Planners and Technologists. A background in IP-based or Ethernet-based management is helpful and familiarity with the target operating system is assumed. This course does not assume RDMA specific knowledge and programming for RDMA is not a prerequisite.
- Register now.

Both events will be held at the University of New Hampshire Interoperability Lab in Durham, NH. Read the Full Story.

The post OpenFabrics Alliance to Host RDMA Training Courses in New Hampshire appeared first on insideHPC.

Revealing Hidden Latency Patterns with Heat Maps

Ralph — Mon, 20 May 2013 16:18:49 +0000

Over at Brendan’s Blog, Brendan Gregg writes that response time – or latency – is crucial to understand in detail, but many of the common presentations of this data hide important details and patterns.

When I/O latency is presented as a visual heat map, some intriguing and beautiful patterns can emerge. These patterns provide insight into how a system is actually performing and what kinds of latency end-user applications experience. Many characteristics seen in these patterns are still not understood, but so far their analysis is revealing systemic behaviors that were previously unknown.

Read the Full Story.

The post Revealing Hidden Latency Patterns with Heat Maps appeared first on insideHPC.

Hybrid Computing Trends call for Code Portability

Rich Brueckner — Wed, 15 May 2013 14:23:06 +0000

Over at the Xcelerit Blog, Jörg Lotze and Hicham Lahlou write that code portability is the key to success in a hybrid computing world with so many available processing architectures.

Therefore, often compromises are taken: typically easy maintenance is favoured and performance is sacrificed. That is, the code is not optimised for a particular platform and developed for a standard CPU processor, as maintaining code bases for different accelerator processors is a difficult task and the benefit is not known beforehand or does not justify the effort. The best solution however would be a single code base that is easy to maintain, written in such a way that it can run on a wide variety of hardware platforms – for example using the Xcelerit SDK. This allows to exploit hybrid hardware configurations to the best advantage and is portable to future platforms.

Read the Full Story.

The post Hybrid Computing Trends call for Code Portability appeared first on insideHPC.

ScaleMP Using RAM plus vSMP to Boost Server Performance

Rich Brueckner — Mon, 13 May 2013 15:21:21 +0000

ScaleMP uses is vSMP aggregation hypervisor and cheap skinny nodes to boost main memory

Over at The Register, Timothy Prickett-Morgan writes that ScaleMP has cooked up some new ways to build big memory machines for less money.

In the example above, the workload in question only needs a four-socket Xeon E5-4600 or Xeon E7-4800 in terms of the processing capacity, but the 48 to 64 memory sticks in this box do not offer enough main memory capacity, and moreover, the fat memory needed to build up terabytes of memory space are very expensive. So instead of buying an eight-socket box to get more memory slots, you get the four-socket box and put in the faster Xeon or Opteron processors you can afford. Then you buy a bunch of skinny server nodes with 24 memory sticks each, and you turn off the cores and leave on the memory controllers and memory in the boxes as well as the InfiniBand ports, and now the FDR links are effectively a backplane for an SMP based on the vSMP hypervisor.

If this sounds enticing, the good news is that there will soon be an easy way to kick the tires and give it a try. Morgan also writes that ScaleMP plans to roll out a variant of vSMP Foundation called Memory Expansion Free. As a free download without support, Memory Expansion Free will have one compute node and up to a total of eight nodes in a cluster and is also limited to four sockets of processing in a machine and 1TB of aggregate main memory across the server nodes in the cluster.

Read the Full Story.

The post ScaleMP Using RAM plus vSMP to Boost Server Performance appeared first on insideHPC.

Performance & Functionality Testbed for Clustered Filesystems at Bank of Italy

Rich Brueckner — Sat, 11 May 2013 17:48:35 +0000

In this video from the Lustre User Group 2013, Giuseppe Bruno from the Bank of Italy presents: Performance & Functionality Testbed for Clustered Filesystems.

Download the slides (PDF) or check out more Lustre presentations in our LUG 2013 Video Gallery.

The post Performance & Functionality Testbed for Clustered Filesystems at Bank of Italy appeared first on insideHPC.

A Look at Five HPC Buzzwords

Ralph — Sat, 11 May 2013 14:47:38 +0000

Over at the Adaptive Computing Blog, Ian Nate writes that five HPC buzwords are dominating the conversation as of late.

We’ve noticed a rise in the use of Energy-Efficient Computing, especially when it comes to HPC and Datacenter. A key factor in the future of large-scale HPC systems, energy efficiency is emerging as likely a second big obstacle to reaching exascale. The reason is the cost of powering an exascale system is exponentially higher than the current petascale systems, and power isn’t getting any cheaper. As an industry, HPC will have to weigh the benefits, or need, for exascale, versus the cost to house and power such systems. The good news is that many systems in Europe are already thinking green because of the higher energy costs. And, we’re seeing a stronger presence for Green Computing in the United States, with systems like NICS’ Beacon reaching the top of the Green500, a list that has picked up significant steam since its initial release in 2007.

The other buzzwords include Big Data, Exascale, Petascale Race, and HPC Cloud. Read the Full Story.

The post A Look at Five HPC Buzzwords appeared first on insideHPC.

Video: Managing and Monitoring a Scalable Lustre Infrastructure

Rich Brueckner — Fri, 10 May 2013 16:06:00 +0000

In this video from the Lustre User Group 2013, Makia Minich from Xyratex presents: Managing and Monitoring a Scalable Lustre Infrastructure. Download the slides (PDF) or check out our LUG 2013 Video Gallery.

During his talk, Makia mentions an excellent presentation from John West entitled What’s Missing from HPC.

The post Video: Managing and Monitoring a Scalable Lustre Infrastructure appeared first on insideHPC.

Jeff Layton on Tools for Data Migration

Rich Brueckner — Fri, 10 May 2013 15:37:29 +0000

Over at HPC Admin, Dell’s Jeff Layton writes that with today’s explosive data growth, at some point you will have to migrate data from one set of storage devices to another. To help move things along, he provides an overview of data migration tools.

At some point during this growth spurt, you will have to think about migrating your data from an old storage solution to a new one, but copying the data over isn’t as easy as it sounds. You would like to preserve the attributes of the data during the migration, including xattrs (extended attributes), and losing information such as file ownership or timestamps can cause havoc with projects. Plus, you have to pay attention to the same things for directories; they are just as important as the file themselves (remember that everything is a file in Linux). In this article, I wanted to present some possible tools for helping with data migration, and I covered just a few of them. However, I also wanted to take a few paragraphs to emphasize that you need to plan your data migration if you want to succeed.

As referred to in Layton’s article, this video from the Lustre User Group 2013 features Marc Stearman from LLNL presenting: Sequoia Data Migration Experiences. Download the slides (PDF).

Read the Full Story.

The post Jeff Layton on Tools for Data Migration appeared first on insideHPC.

Video: Aeon Computing Showcases Intelligent Disk Arrays with Lustre and ZFS

Rich Brueckner — Fri, 10 May 2013 11:30:52 +0000

In this video from the Lustre User Group 2013, Doug Johnson from Aeon Computing describes the company’s Hydra storage arrays that marry the speed of Lustre with the HA capabilities of ZFS.

The Hydra60 is a combination Lustre OSS (object storage server) and OST (object storage target) with two active/active failover nodes and shared storage in a single system chassis with an ultra dense 60 drive 6Gb SAS storage infrastructure. With a unified and zonable 6Gb SAS dual-ported backplane and drives the Hydra60 can sustain a remarkable performance while providing high-availability to volumes or object storage. With external interface options including FDR Infiniband, 40/10GbE 1Gb Ethernet and supporting Linux and Lustre releases 2.x the Hydra60 makes an excellent storage platform for Lustre performance with HA operation. The design of Hydra60 provides an affordable, redundant and resilient storage platform by leveraging RAIDZ thereby eliminating the cost of hardware RAID controller technology.”

For more on Lustre, check out our LUG 2013 Video Gallery.

The post Video: Aeon Computing Showcases Intelligent Disk Arrays with Lustre and ZFS appeared first on insideHPC.

Slidecast: ClusterStor 1500 Departmental Scale-Out Storage for HPC

Rich Brueckner — Thu, 09 May 2013 14:32:21 +0000

In this slidecast, Ken Claffey from Xyratex describes the company’s new ClusterStor 1500 storage system. Designed for scale-out HPC storage solutions, the ClusterStor 1500 delivers HPC performance and efficiency with help from the Lustre file system.

Departments within larger organizations or medium-sized enterprises today, especially in the commercial, academic and government sectors, represent an underserved market. They need high-performance and scalable storage solutions that are cost-efficient, easy to deploy and manage and reliable even under heavy workloads,” said Ken Claffey, senior vice president of the ClusterStor business at Xyratex. “Growth in this market segment is being driven by the increasing adoption of simulation applications in a wide range of industries from car and aircraft design to chemical interactions and financial modeling. Traditional enterprise storage systems are simply not designed to meet the performance needs of these applications, so we engineered and built the affordable and modular ClusterStor 1500 to bring the performance power of Lustre to this underserved and growing market in the way that only ClusterStor can.”

With the ability to scale performance from 1.25GB/s to 110GB/s and raw capacity from 42TB to 7.3PB, ClusterStor 1500 is purpose-built to satisfy data intensive department level compute cluster needs, ClusterStor 1500 is designed to provide best in class scale-out storage for middle tier high performance computing environments. The ClusterStor 1500 solution features scale-out storage building blocks, the Lustre parallel filesystem and a comprehensive management platform that eliminates the guesswork usually associated with building and optimizing your own HPC storage solution.

Read the Full Story * View the slides * Download the MP3 * Subscribe on iTunes * Subscribe to RSS

The post Slidecast: ClusterStor 1500 Departmental Scale-Out Storage for HPC appeared first on insideHPC.

Video: Software vs. Hardware RAID and Implications for the Future

Rich Brueckner — Thu, 09 May 2013 11:00:40 +0000

In this video from the Lustre User Group 2013, Alan Poston from Xyratex presents: Software vs. Hardware RAID and Implications for the Future.

Download the slides (PDF) or check out our LUG 2013 Video Gallery.

The post Video: Software vs. Hardware RAID and Implications for the Future appeared first on insideHPC.

Video: Lessons Learned from the Blue Waters 1 Terabyte/sec File System

Rich Brueckner — Wed, 08 May 2013 13:27:51 +0000

Now that the deployment of the 1 Terabyte/sec file system at Blue Waters has been completed, what comes next? In this video from the Xyratex Blog, John Fragalla, principal solutions architect at Xyratex, discusses the value that ClusterStor brings to the HPC market and what the company has learned from designing and deploying ClusterStor solutions.

The post Video: Lessons Learned from the Blue Waters 1 Terabyte/sec File System appeared first on insideHPC.

Video: HPCS I/O Scenarios Update

Rich Brueckner — Tue, 07 May 2013 16:57:34 +0000

In this video from the Lustre User Group 2013, John Carrier from Cray presents: HPCS I/O Scenarios Update.

Download the slides (PDF) or check out our LUG 2013 Video Gallery.

The post Video: HPCS I/O Scenarios Update appeared first on insideHPC.

Video: European Open Filesystem Update from LUG 2013

Rich Brueckner — Tue, 07 May 2013 14:46:00 +0000

In this video from the Lustre User Group 2013, Hugo Falter from ParTec presents: LUG2013 EOFS Update. As Director of the EOFS Administrative Council, Falter provides an excellent overview on what’s going on with European supercomputing initiatives.

Download the slides (PDF) or check out our LUG 2013 Video Gallery.

The post Video: European Open Filesystem Update from LUG 2013 appeared first on insideHPC.

Video: High Availability in Lustre

Rich Brueckner — Mon, 06 May 2013 14:50:56 +0000

In this video from the Lustre User Group 2013, John Fragalla from Xyratex presents: High Availability in Lustre.

Download the slides (PDF) or check out our LUG 2013 Video Gallery.

The post Video: High Availability in Lustre appeared first on insideHPC.

Video: EIOW Exascale I/O Working Group

Rich Brueckner — Sun, 05 May 2013 15:06:29 +0000

In this video from the Lustre User Group 2013, Meghan McClelland from Xyratex presents: EIOW – Exascale I/O Working Group.

There is a fierce competition on the storage market to offer the best performing devices, with great management at a low price. The EIOW group, from the outset, decided that it would not attempt to offer an end-to-end solution, which would necessarily involve competing instead of working with storage providers. The focus of EIOW is on middleware to provide, for example, schemas describing data structure and layout, novel access methods to data for applications, a uniform data management infrastructure and a framework for the implementation of layered I/O software, similar in spirit to HDF5 as a specialized use of a parallel file system. We decided EIOW should be open, and have interfaces to layer on lower level storage infrastructure such as object stores, databases and file systems as provided by storage providers, to allow their expertise and leadership in this area to continue to benefit the HPC community.

Download the EIOW whitepaper and slides, or check out our LUG 2013 Video Gallery.

The post Video: EIOW Exascale I/O Working Group appeared first on insideHPC.

Video: Panel Discussion on Scaling with PGAS Languages

Rich Brueckner — Sat, 04 May 2013 15:19:21 +0000

In this video from the 2013 Open Fabrics Developer Workshop, Paul Grun from Cray leads a panel discussion on Scaling with PGAS Languages.

Panelists:

Howard Pritchard, Cray (slides)
Mirko Rahn, Fraunhofer Institute
DK Panda, OSU (slides)
Rich Graham, Mellanox
Robert Woodruff, Intel

You can check out more OFA videos at our Open Fabrics Workshop Video Gallery.

The post Video: Panel Discussion on Scaling with PGAS Languages appeared first on insideHPC.

Video: Challenges of Scale & MPI/PGAS APIs

Rich Brueckner — Fri, 03 May 2013 11:30:20 +0000

In this video from the 2013 Open Fabrics Developer Workshop, Todd Rimmer from Intel presents: Challenges of Scale & MPI/PGAS APIs.

Download the slides (PDF). You can check out more OFA videos at our Open Fabrics Workshop Video Gallery.

The post Video: Challenges of Scale & MPI/PGAS APIs appeared first on insideHPC.

Intel MPI Library for Intel Xeon Phi-based Clusters & MVAPICH2 for Intel MIC

Rich Brueckner — Thu, 02 May 2013 16:18:39 +0000

In this video from the 2013 Open Fabrics Developer Workshop, Bill Magro from Intel presents: Intel MPI Library: Implementation for Intel Xeon Phi Based Clusters. Download the slides (PDF).

In this follow-up session, DK Panda from Ohio State University presents: MVAPICH2 for Intel MIC.

You can check out more OFA videos at our Open Fabrics Workshop Video Gallery.

The post Intel MPI Library for Intel Xeon Phi-based Clusters & MVAPICH2 for Intel MIC appeared first on insideHPC.

Video: Details of the Xyratex Acquisition of Lustre Assets

Rich Brueckner — Thu, 02 May 2013 14:23:35 +0000

In this video from the Lustre User Group 2013, Kevin Canady from Xyratex explains the company’s plans for Lustre assets acquired recently from Oracle.

Download the slides (PDF). You can check out more Lustre presentations at our LUG 2013 Video Gallery.

The post Video: Details of the Xyratex Acquisition of Lustre Assets appeared first on insideHPC.