“This talk will present RDMA-based designs using OpenFabrics Verbs and heterogeneous storage architectures to accelerate multiple components of Hadoop (HDFS, MapReduce, RPC, and HBase), Spark and Memcached. An overview of the associated RDMAenabled software libraries being designed and publicly distributed as a part of the HiBD project.”
“Group Deutsche Boerse is a global financial service organization covering the entire value chain from trading, market data, clearing, settlement to custody. While reliability has been a fundamental requirement for exchanges since the introduction of electronic trading systems in the 1990s, since about 10 years also low and predictable latency of the entire system has become a major design objective. Both issues have been important architecture considerations, when Deutsche Boerse started to develop an entirely new derivatives trading system T7 for its options market in the US (ISE) in 2008. As the best fit at the time a combination of InfiniBand with IBM WebSphere MQ Low Latency Messaging (WLLM) as the messaging solution was determined. Since then the same system has been adopted for EUREX, one of the largest derivatives exchanges in the world, and is now also extended to cover cash markets. The session will present the design of the application and its interdependence with the combination of InfiniBand and WLLM. Also practical experiences with InfiniBand in the last couple of years will be reflected upon.”
Today One Stop Systems announced the results of High-Tech Online Magazine Tom’s IT PRO’s review of the 200TB Flash Storage Array SAN (FSA-SAN). OSS’ FSA-SAN provides 200TB of shared Flash RAIDed memory with a choice of the fastest SAN connectivity options including Fiber Channel, Infiniband, and iSCSI.
Today Scality announced the production deployment of the Scality RING to power Los Alamo National Laboratory’s Trinity supercomputer, projected to be one of the world’s fastest. Trinity, part of the NNSA Advanced Simulation and Computing Program, is expected to be the first platform large and fast enough to begin to accommodate finely resolved 3D calculations for mission-critical simulations. As part of the deployment, Scality is also working together with Los Alamos on MarFS, an open source software project that brings the power of object storage to all large-scale research computing environments, including the U.S. Department of Energy.
A design for virtual Ethernet networks over IB is described. The virtual Ethernet networks are implemented as overlays on the IB network. They are managed in a flexible manner using software. Virtual networks can be created, removed, and assigned to servers dynamically using this software. A virtual network can exist entirely on the IB fabric, or it can have an uplink connecting it to physical Ethernet using a gateway. The virtual networks are represented on the servers by virtual network interfaces which can be used with para-virtualized I/O, SRIOV,and non-virtualized I/O. This technology has many uses: communication between applications which are not IB-aware, communication between IB-connected servers and Ethernet-connected servers, and multi-tenancy for cloud environments. It can be used in conjunction with OpenStack, such as for tenant networks.”
This week, Molex announced the acquisition of Interconnect Systems, Inc. (ISI) which specializes in the design and manufacture of high density silicon packaging with advanced interconnect technologies. “We are thrilled to join forces with Molex. By combining respective strengths and leveraging their global manufacturing footprint, we can more efficiently and effectively provide customers with advanced technology platforms and top-notch support services, while scaling up to higher volume production,” said Bill Miller, president, ISI.
NVMe over Fabrics is an enhancement / expansion of the NVMe specification to support remote access to NVMe storage resources via RDMA. This presentation will review the major components of the NVMe over Fabrics specification, describe the host and target stack, discuss the plan for Linux open source drivers, and preview some performance characteristics. The discussion will also touch on possible deployment models.
In this special guest feature, Ken Strandberg offers this live report from Day 2 of the Lustre User Group meeting in Portland. “Scott Yockel from Harvard University shared how they are deploying Lustre across their massive three data centers up to 90 miles apart with 25 PB of storage, about half of which is Lustre. They’re using Docker containers and employing a backup strategy across the miles of every NFS system, parsing of the entire MDT, and includes 10k directories of small files.”
“The first release of OpenFabrics Interfaces (OFI) software, libfabric, occurred in January of 2015. Since then, the number of fabrics and applications supported by OFI has increased, with considerable industry momentum building behind it. This talk discusses the current state of OFI, then speaks to the application requirements driving it forward. It will detail the fabrics supported by libfabric, identify applications which have ported to it, and outline future enhancements to the libfabric interfaces and architecture.”
Today Egypt’s Bibliotheca Alexandrina library announced plans to build an HPC platform using Huawei technologies. Based on high-density FusionServer servers, the 118 Teraflop Huawei cluster employs high-speed InfiniBand and 288 TB in storage capacity for concurrent file systems.