Over the past several years, virtualization has made major inroads into enterprise IT infrastructures. And now it is moving into the realm of high performance computing (HPC), especially for such compute intensive applications as electronic design automation (EDA), life sciences, financial services and digital media entertainment. This article is the first in a series that explores the benefits the HPC community can achieve by adopting proven virtualization and cloud technologies. The complete insideHPC Guide to Virtualization, the Cloud and HPC is available for download from the insideHPC White Paper Library.
Introduction to Virtualization for HPC
It took a while. Virtualization dates back to the 1960s when some of the earliest mainframes claimed center stage in the “glass house” data centers of the day. Back then the term meant dividing up mainframe’s resources to handle various applications. However, it wasn’t until the late 1990s when a group of Stanford University researchers figured out how to virtualize the x86 instruction set that virtualization was able to move from the mainframe to the mainstream. Those same researchers went on to form VMware and release the first virtualization product for the x86 ISA in 1999.
What is Virtualization?
Hardware virtualization refers to the creation of a number of self-contained virtual servers that are resident on the physical server, or host machine. Virtualization allows multiple applications to be run on the same machine while providing security and fault isolation. Typically an administrator decides how much of each resource – CPU, memory, networking – to allocate to the virtual machine (VM), while assigning priorities to different classes of users. The virtual infrastructure dynamically enforces these policies to ensure that each VM gets its fair share of resources.
Another key attribute of x86 virtualization important to HPC is hardware abstraction. With virtualization, there is no need to retool the IT infrastructure’s OS and drivers every time new hardware is added to a cluster. The OS always sees the same virtual hardware regardless of the underlying physical hardware type. Hardware abstraction provides the ease of use and agility that physical (bare metal) clusters lack.
As virtualization became more prevalent, both Intel and AMD have continued to add more hardware support for the technology, including CPU, memory and I/O virtualization. These hardware advancements, coupled with increasingly sophisticated virtualization software, have resulted in improved performance for a wide variety of workloads, including HPC-based applications.
Server virtualization used in the enterprise data center for IT infrastructure workload has yielded a number of benefits.
Some typical benefits include:
- Significantly increases server utilization
- Improves business agility by providing the additional computational resources that enable companies to bring products to market faster and be more competitive in global markets
- Provides significant reductions in CapEx and OpEx for each server virtualized
- Highly resilient in dealing with power outages, equipment failure or other disruptions
- Enhances security
Virtualizing Networking Services
With the development of software-defined networks (SDN) over the past decade, the principles of virtualization have been applied to network resources – abstracting, pooling and automating them to overcome the limits of rigid physical architecture.
Network services are assigned to each application and elastically adapt to meet the application’s changing requirements. The choreography between the compute and network virtualization provides an unparalleled degree of automation, security, flexibility and agility.
Security in a Virtualized Environment
Security has turned out to be one of the major benefits associated with virtualization. It is delivered as a software-defined service that is decoupled from physical devices, aggregated, and targeted to wherever it is needed without requiring a hardware upgrade. This allows workloads to be scaled and moved without security constraints or the need for specialized appliances.
VMware’s approach uses “bare metal” virtualization – the hypervisor interfaces directly with the computer hardware without the need for a host operating system. This safeguards the virtualization software from OS-related vulnerabilities, making the VMware platform one of the most secure in the industry.
Integrated firewalling and optimized gateway services protect the network edge. And VMware’s open architecture allows data center managers to integrate existing security investments and the latest third party innovations.
Over the next few weeks we will explore these HPC Virtualization topics:
- Challenges on the Road to Virtualization
- Virtualization and Secure Private Cloud
- Workload Agility in the HPC Cluster
- Software Defined Data Center
- Virtualization in the Cloud
- Convergence of HPC and the Enterprise
If you prefer the complete insideHPC Guide to Virtualization, the Cloud and HPC is available for download in PDF from the insideHPC White Paper Library, courtesy of VMware.
Great article! This is not only needed for cost control, immediate access, and security and there are other benefits. I see the trend going towards Technical Computing Marketplaces (like Nimbis Services) where a user can sporadically use an application, which opens up technical computing and HPC applications to the group of users that couldn’t afford it otherwise. Also, Remote Visualization (like NICE Software) where users can save enormous amounts of time in collaboration.