If you open the back of today’s HPC cluster you will see lots of cables. There are any number of cables including those for power, Ethernet, InfiniBand , Fibre Channel, KVM, and others. The current situation creates the need for complex configuration and administration. Managing the complexity of these HPC systems has been made easier with tools like Bright Cluster Manager where the administrator works from a “single pane” interface to the whole cluster.
The first Beowulf HPC clusters were built using Ethernet. This decision was primarily driven by the low cost and high availability of commodity Ethernet technology. The low cost of Ethernet was due to ubiquitous use in areas of IT other than HPC. In the early days of cluster based HPC, exploiting the economics of the larger market share was the most cost efficient way to build systems.
Outside of HPC, Ethernet enjoyed great adoption and growth and became the standard interconnect for data centers everywhere. In particular, as each successive generation was deployed in large volumes, prices dropped further facilitating market uptake. Currently 10 and 40 Gigabit Ethernet (GbE) technologies are becoming adopted at ever increasing rates across all market verticals throughout the IT. Ethernet also has a plug-and-play reputation that spans multiple generations of compatibility.
One of the biggest complaints about Ethernet as and HPC interconnect has been sub-par performance when compared to InfiniBand. Of course InfiniBand benefits from “user space” or “kernel bypass” methods that greatly improve latency and throughput of the interconnect. This situation has changed with the introduction of high performance Ethernet.
Cisco’s recently developed high-speed low-latency interconnect technology known as usNIC (user-space NIC) achieved out-of-the-box (without any software or hardware optimization) 89.69% HPL efficiency across 512 Intel Xeon Cores.
As shown in a recent white paper by Bright Computing, the performance of Cisco UCS clusters equipped with the usNIC can match and even exceed InfiniBand performance. Also shown in the white paper are surprising low Open MPI latencies and excellent benchmark performance of real-world applications.
Interestingly, Cisco UCS systems are designed with a unified fabric that condenses three network types—IP, storage, and management—into one. This approach means that the system is wired only once, when it is installed, with bandwidth allocations and I/O configurations managed dynamically.
To help with HPC management, Bright Cluster Manager 7.0 can configure, monitor, and manage Cisco UCS C-Series rack servers directly either in a standalone mode or via the UCS Manager. Deployment and management of UCS C-Series servers from Bright Cluster Manager is designed to provide comprehensive control and manageability for all HPC related administration tasks.
Details of how the combination of Bright Cluster Manager and Cisco UCS servers break the existing rules of cluster design and provide an entirely new level of integration, management, and performance previously unavailable in the HPC market are explored in the white paper. The highly desirable “all Ethernet HPC cluster” has become an extremely viable solution for technical computing offering the following advantages.
- True HPC performance over 10 and 40 Gigabit Ethernet
- Reduced cost and complexity through the ability to consolidate the number of network fabrics (IB, FC) onto a single high performance Ethernet network.
- Complete and cost saving HPC administration from Bright Cluster Manager
- Deep management of all cluster components.
- Easy node provisioning without moving cables.
Bright Cluster Manger running on Cisco UCS systems offers a complete HPC Ethernet solution that allows the IT manager, administrator, and user to focus on HPC results and not the minutia of managing a complex collection of hardware.
Get this and other HPC white papers from the insideHPC White Paper Library.
