Sign up for our newsletter and get the latest HPC news and analysis.

What is the Virtual Interface Architecture?

The VIA is an abstract model of a user-level zero-copy network, and is the basis for InfiniBand and iWARP.

Networks are a shared resource. In traditional networks such as Ethernet, the network is protected by the kernel, which presents a tremendous performance bottleneck when latency is an issue.

One of the classic developments in computing systems is virtual memory, a combination of hardware and software that creates the illusion of private memory for each process. In the same school of thought, a virtual network interface protected across process boundaries could be accessed at the user level. With this technology, the “consumer” manages his own buffers and communication schedule while the “provider” handles the protection.

Thus, the NIC provides a “private network” for a process, and a process is usually allowed to have multiple such networks. The virtual interface (VI) of VIA refers to this network and is merely the destination of the user’s communication requests. Communication takes place over a pair of VIs, one on each of the processing nodes involved in the transmission. In “kernel-bypass” communication, the user manages his own buffers.

Another facet of traditional networks is that arriving data is placed in a pre-allocated buffer and then copied to the user-specified final destination. Copying large messages can take a long time, and so eliminating this step is beneficial. Another classic development in computing systems is direct memory access (DMA), in which a device can access main memory directly while the CPU is free to perform other tasks.

In a network with “remote direct memory access” (RDMA), the sending NIC uses DMA to read data in the user-specified buffer and transmit it as a self-contained message across the network. The receiving NIC then uses DMA to place the data into the user-specified buffer. There is no intermediary copying and all of these actions occur without involvement of the CPUs, which has an added benefit of lower CPU utilization.

For the NIC to actually access the data through DMA, the user’s page must be in memory. In VIA, the user must “pin-down” his buffers before transmission, so as to prevent the OS from swapping the page out to the disk. This action—one of the few that involves the kernel—ties the page to physical memory. To ensure that only the process that owns the registered memory may access it, the VIA NICs require permission keys known as “protection tags” during communication.

So essentially VIA is a standard that defines kernel bypassing and RDMA in a network. It also defines a programming library called “VIPL”. It has been implemented, most notably in cLAN from Giganet (now Emulex). Mostly though, VIA’s major contribution has been in providing a basis for InfiniBand and iWARP.

Resource Links: