Sign up for our newsletter and get the latest big data news and analysis.
Daily
Weekly

Using High Performance Interconnects in Dynamic Environments

“Over the last years the OFA community has shown the potential of using high performance networks (InfiniBand) to boost the performance of virtualized cloud environments, however, the network reconfiguration challenges still continue to exist. In this session we present the work we have been doing on InfiniBand subnet management and routing, in the context of dynamic cloud environments. This work includes, but not limited to, techniques in order to provide better management scalability when virtual machines are live migrating, tenant network isolation in multi-tenant environments, and fast performance-driven network reconfiguration.”

Video: Exploiting HPC Technologies to Accelerate Big Data Processing

“This talk will present RDMA-based designs using OpenFabrics Verbs and heterogeneous storage architectures to accelerate multiple components of Hadoop (HDFS, MapReduce, RPC, and HBase), Spark and Memcached. An overview of the associated RDMAenabled software libraries being designed and publicly distributed as a part of the HiBD project.”

Video: Software-Defined Networking on InfiniBand Fabrics

A design for virtual Ethernet networks over IB is described. The virtual Ethernet networks are implemented as overlays on the IB network. They are managed in a flexible manner using software. Virtual networks can be created, removed, and assigned to servers dynamically using this software. A virtual network can exist entirely on the IB fabric, or it can have an uplink connecting it to physical Ethernet using a gateway. The virtual networks are represented on the servers by virtual network interfaces which can be used with para-virtualized I/O, SRIOV,and non-virtualized I/O. This technology has many uses: communication between applications which are not IB-aware, communication between IB-connected servers and Ethernet-connected servers, and multi-tenancy for cloud environments. It can be used in conjunction with OpenStack, such as for tenant networks.”

Video: Past, Present, and Future of OpenFabrics Interfaces

“The first release of OpenFabrics Interfaces (OFI) software, libfabric, occurred in January of 2015. Since then, the number of fabrics and applications supported by OFI has increased, with considerable industry momentum building behind it. This talk discusses the current state of OFI, then speaks to the application requirements driving it forward. It will detail the fabrics supported by libfabric, identify applications which have ported to it, and outline future enhancements to the libfabric interfaces and architecture.”

Gary Grider Presents: HPC Storage and IO Trends and Workflows

“Trends in computer memory/storage technology are in flux perhaps more so now than in the last two decades. Economic analysis of HPC storage hierarchies has led to new tiers of storage being added to the next fleet of supercomputers including Burst Buffers or In-System Solid State Storage and Campaign Storage. This talk will cover the background that brought us these new storage tiers and postulate what the economic crystal ball looks like for the coming decade. Further it will suggest methods of leveraging HPC workflow studies to inform the continued evolution of the HPC storage hierarchy.”

Gary Grider from LANL to Keynote OpenFabrics Workshop

Gary Grider from Los Alamos National Laboratory will keynote the 2016 OpenFabrics Workshop this year with a talk on HPC Storage and IO Trends and Workflows. The event takes place April 4-8, 2016 in Monterey, California.

Video: A Brief Introduction to OpenFabrics

Sean Hefty from Intel presented this talk at the Intel HPC Developer Conference at SC15. “OpenFabrics Interfaces (OFI) is a framework focused on exporting fabric communication services to applications. OFI is best described as a collection of libraries and applications used to export fabric services. The key components of OFI are: application interfaces, provider libraries, kernel services, daemons, and test applications. Libfabric is a core component of OFI. It is the library that defines and exports the user-space API of OFI, and is typically the only software that applications deal with directly. It works in conjunction with provider libraries, which are often integrated directly into libfabric.”

Call for Participation: 2016 Open Fabrics Workshop

The 2016 OpenFabrics Workshop has issued its Call for Participation. The event takes place April 4-8, 2016 in Monterey, California. The Workshop is the premier event for collaboration between OpenFabrics Software (OFS) producers and those whose systems and applications depend on the technology. Every year, the workshop generates lively exchanges among Alliance members, developers and users who all share a vested interest in high performance networks.

Video: Workload Driven Memory Registration

In this video from the 2015 OFS Developer’s Workshop, Parav Pandit from Emulex presents: Workload Driven Memory Registration.

Video: NVM Express Introduction and Tutorial

NVM Express is a specification for accessing solid-state drives (SSDs) attached through the PCI Express (PCIe) bus. “NVM” stands as an acronym for non-volatile memory, which is used in SSDs. As a logical device interface, NVM Express has been designed from the ground up, capitalizing on the low latency and parallelism of PCI Express SSDs, and mirroring the parallelism of contemporary CPUs, platforms and applications. By allowing parallelism levels offered by SSDs to be fully utilized by host’s hardware and software, NVM Express brings various performance improvements.