In many AI applications, transporting large amounts of data back to a remote datacenter is impractical and undesirable. This sponsored post from One Stop Systems’ Tim Miller highlights how with AI on the Fly, the entire AI workflow resides at the edge at the data source for high speed data capture.
Tim Miller, Vice President Strategic Development at OSS
AI on the Fly™ puts computing and storage resources for the entire AI workflow, not in the datacenter, but on the edge near the sources of data. Applications are emerging for this new AI paradigm in diverse areas including autonomous vehicles, predictive personalized medicine, battlefield command and control, and industrial automation. The common elements of these solutions are high data rate acquisition, high speed low latency storage and efficient high performance AI training and inference computing. All of these building block elements are connected seamlessly with memory mapped PCI Express system, interconnected and customized as appropriate, to meet the specific environmental requirements of ‘in the field’ installations.
At the front end of these systems is high speed data acquisition technology. Depending on the application, the data can be generated from a wide array of sensors. In the case of autonomous vehicles, data is generated through arrays of video and LIDAR sensors. In battlefield applications radar, sonar, FLIR (infrared), and RF sensors are deployed. Medical applications use MRI or CT sensors. In security applications, networks of security cameras produce high volumes of video data. Industrial automation includes telemetry data from IoT sensors and video feeds at a wide spectrum of frame rates.
Figure 1. AI on the Fly Workflow
Although data rates vary for all of these applications based on the sensors or array of sensors deployed, the fundamental requirements for the ingest subsystem is that it supports high speed data rates, does not allow loss of data, and does not impose flow control on the sensor data stream. For many AI on the Fly applications, local data rates can be extremely high, as much as 100 Gbps per stream or more, requiring specialized PCIe data capture hardware. As part of the capture process the data is often processed in real time to be formatted in a useable form prior to movement to the storage devices. (See Figure 1: Data Clean & Prep). Capture hardware can be in the form of PCIe FPGAs, video capture (encoded and raw), frame grabbers, or smart NICs performing a range of functions including tagging, encoding, sorting, analog to digital conversion, filtering, time stamping, and channel synchronization. The data rates of the acquisition front-end drive high performance requirements for the storage subsystem, necessitating the use of direct PCIe attached NVMe SSD storage devices. In these systems, storage needs to be able to scale to potentially 10’s or 100’s of TBs. Additional storage subsystem requirements include high availability/redundancy, security and optional support for removable storage media.
Few companies have the range of expertise to develop and deploy these edge-focused data acquisition systems in support of AI workflows. One Stop Systems (OSS) with expertise in large scale high performance specialized PCI Express, NVMe storage and AI-system architecture is one such company. OSS recently announced a flexible AI data acquisition platform based on end-to-end PCIe Gen 4 using AMD’s latest generation EPYC™ 7002 processor servers and Gen 4 NVMe SSDs to achieve 56GB/s data ingress capability. The entire solution includes a 1U AMD 2nd generation EPYC 7002 processor based server and an OSS 4UV PCIe Gen 4 scale-out expansion system with up to 16 32TB NVMe cards. The expansion system directly attaches to the server with two OSS PCIe x16 Gen 4 links providing 512 gbps bandwidth and can provide up to 512TB storage capacity. Variations of the system configuration can support up to 8 x16 PCIe Gen 4 data acquisition cards.
A representative example of AI on the Fly data acquisition is autonomous vehicle development fleets. OSS recently announced a collaborative engineering design win with a major international network transportation company for deployment of AI on the Fly components in its 150 vehicle autonomous driving development fleet. This fleet is being used to gather the data required to develop and test the artificial intelligence algorithms which will eventually be deployed in thousands of commercial vehicles. In this case, the AI on the Fly data ingest system is tied to a myriad of video, radar and LIDAR sensors in the car aggregated through redundant 50 gbps Ethernet connections to the storage subsystem, which is then interconnected directly to multi-GPU machine learning training and inference compute engines. The entire system is deployed in the trunk of the automobile.
In many AI applications, transporting large amounts of data back to a remote datacenter is impractical and undesirable. With AI on the Fly the entire AI workflow resides at the edge at the data source. High performance scalable data acquisition is a fundamental and enabling component of this emerging new paradigm.
Tim Miller is Vice President of Strategic Development at One Stop Systems.
Disclaimer: This article may contain forward-looking statements based on One Stop Systems’ current expectations and assumptions regarding the company’s business and the performance of its products, the economy and other future conditions and forecasts of future events, circumstances and results.
