Sign up for our newsletter and get the latest HPC news and analysis.
Send me information from insideHPC:


2019: The Year of PCI Express 4.0

In this guest feature, Tim Miller, Vice President Strategic Development for One Stop Systems, explores the expected speed and innovation stemming from the introduction of PCI Express 4.0. 

high performance embedded computing

Tim Miller, Vice President Strategic Development, One Stop Systems

Computer systems are about to get a whole lot faster. For almost two decades PCI Express has been the data interconnect standard knitting together CPU, storage, networking, IO and GPUs in PCs to High Performance Computing (HPC) systems. In that timeframe it has delivered a doubling of bandwidth every three years on average. Today, virtually all computer systems are based on PCI Express 3.0 delivering 8GT (gigatransfers per second) per lane resulting in 32GB/sec bandwidth in a standard 16 lane connection (duplex). This year, starting at the high end of the market, a transition will begin toward systems based on PCI Express 4.0. The interconnect speed will double to 64GB/sec in a 16 lane connection. The speed increase will provide a welcome improvement across the full range of HPC applications such as AI/machine learning, scientific simulation, big data analytics, high resolution visualization and rendering, and genomics analysis.

As with each generational transition, the ecosystem of technology vendors adopts the new standard and begins to deliver compliant products – but generally on different timelines. OEMs typically need to wait for the critical mass of technology components to be available prior to developing their platforms and solutions. The first critical enablers are the CPU and chipset silicon from the major processor vendors. IBM Power and AMD x86 and some ARM implementations are available now. The Intel processors are expected in 2020. Other key ecosystem pieces including high speed networking interface cards including 100gbps Ethernet and EDR Infiniband and NVMe SSD are available now or expected shortly. Although some GPUs and other accelerators with 4.0 interfaces are expected shortly, the NVidia GPUs are not expected until 2020. An important element for scaling systems is the PCI Express 4.0 switches, which are also available now from Broadcom and Microsemi.

Although 2019 will be the year of PCI Express 4.0 the upward march in performance will continue well into the next decade.

Although adopting this fourth generation is a predictable path forward for mid-tier platform and appliance vendors, few have the expertise and experience to design complex systems utilizing these data rates. Expertise in high speed signal layout and signal integrity and design/debug tools are critical, especially when addressing specialized or large PCI Express fabrics requirements scaling across large backplanes, connectors and even cabling to external chassis. Delivering these specialized high end systems often require partnership between OEMs/system Integrators bringing deep vertical market or application expertise with technology experts who can provide the underlying high performance building blocks and system design expertise.

One Stop Systems (OSS) has provided this critical expertise and delivered enabling system building blocks to the market across each of the PCI Express generation transitions. They have led each of these transitions by delivering the first host interface card/cable solutions supporting external expansion chassis. Gen 4.0 is no exception. OSS announced the PCI Express 4.0 HIB (host interface) in September of 2018. The card allows connection of remote PCI Express 4.0 devices to a host computer. This latest generation adapter fits in a PCI Express 4.0 x16 half-length, half-height server slot, and features four SFF-8644 cable connectors on the IO bracket. The Gen 4.0 adapter complies with the PCI-SIG PCI Express cable standard, and it uses mini-SAS HD and cable management interface (CMI) compliant copper and fiber optic cables. This flexibility allows for the highest speed PCI Express system-to-system and rack-to-rack interconnects in data centers and OEM applications, delivering bandwidth up to 256 Gbps and latency of less than 150 ns.

OSS also recently announced the first PCI Express 4.0 expansion backplanes which feature one Gen 4.0 x16 target slot and either five Gen 4.0 x16 or eight x8 downstream slots. Two of these backplanes can be installed into the OSS 4U Value Chassis along with Gen 4.0 cable adapters and support up to 8 double wide GPUs or 16 NVMe SSD cards to create next generation HPC expansion platforms. The Value Chassis can be connected to any Gen 4.0 equipped server with two 256Gbps connections. Devices with Gen 3.0 interfaces also benefit in this architecture as the chassis aggregates the traffic and provides connection to the server at the full Gen 4.0 speed.

OSS is currently working with a number of customers developing their Gen 4.0 bases solutions and anticipates upgrading their entire product line of multi-GPU and flash storage integrated and expansion systems to Gen 4.0 this year.

Although 2019 will be the year of PCI Express 4.0, the upward march in performance will continue well into the next decade. The PCI SIG is scheduled to releases the PCI Express 5.0 specification shortly doubling interconnect performance again to 128GB/sec for a 16 lane connection. OSS anticipates being at the forefront of this introduction as well providing early adopter OEMs access to PCI Express 5.0 building blocks in 2020.

Tim Miller is Vice President Strategic Development for One Stop Systems

Leave a Comment

*

Resource Links: