insideHPC » Virtualization

Video: The Sofware-defined Datacenter & RDMA in Virtual Environments

Rich Brueckner — Sun, 28 Apr 2013 11:00:21 +0000

In this video from the 2013 Open Fabrics Developer Workshop, VMware’s Josh Simons presents: The Sofware-defined Datacenter.

The software-defined datacenter vision took the industry by storm in 2012. It represents a prescriptive model that brings the benefits of virtualization to the rest of the datacenter. Expect to see the move towards a software-defined datacenter accelerate in 2013. Networking and infrastructure security represent some of the stickiest issues when it comes to the drive to a more agile data center. And because of this strong customer interest in SDDCs, you’ll also see more networking vendors and startups modify their roadmaps to steer towards a software-defined networking strategy.

Read the Full Story or Download the Slides (PDF).

In this follow-up presentation, Bhavesh Davda from VMware gives a talk on RDMA in Virtual Environments.

The post Video: The Sofware-defined Datacenter & RDMA in Virtual Environments appeared first on insideHPC.

How MIT’s StarCluster Powers Virtualization for Cloud HPC

Rich Brueckner — Wed, 30 Jan 2013 16:30:14 +0000

Over at Admin HPC, Gavin W. Burris writes that virtualization has become a viable option for researchers with a need for cluster computing power thanks in part to StarCluster, MIT’s open-source toolkit for launching, controlling, and orchestrating clusters of virtual servers within the Amazon Elastic Compute Cloud (EC2) service. StarCluster provides a number of images pre-rolled and ready to go, as well as a command line tool for wrangling them into functional computing power.

The cloud has become a key resource in the support of HPC. Given the proper use case, cloud offerings are an affordable fit for a variety of different workflows. A key tool in any systems programmer’s arsenal should be the StarCluster toolkit, which provides a powerful interface for harnessing these cloud resources in an effective manner. The toolkit builds from the well-established model of HPC clustering with Linux, providing a clear path to transferring existing skills and code to the cloud.

Read the Full Story.

The post How MIT’s StarCluster Powers Virtualization for Cloud HPC appeared first on insideHPC.

Video: The Growing Need for HPC Virtualization

Rich Brueckner — Tue, 01 Jan 2013 12:00:24 +0000

In this video, Professor Thomas J. Hacker from Purdue University lectures on virtualization clusters in high-performance computing. Additional segments on this subject are also available on YouTube.

The post Video: The Growing Need for HPC Virtualization appeared first on insideHPC.

What Would You Do with 8 Terabytes of Memory?

Rich Brueckner — Wed, 12 Dec 2012 20:16:09 +0000

What would you do with 8 Terabytes of main memory? ScaleMP wants to know, and you could win lots of coffee and a t-shirt for your trouble.

Let us know below with your description and include what application you would run, what problems you are trying to solve and how many cores your application would need. Think outside of the box. Don’t be limited in your description by the small servers you might be using today. Describe what you would do with 8 TB of main memory. Then fill in the contact information below and we will send you either a free T-shirt or Starbucks gift card.

Enter now.

The post What Would You Do with 8 Terabytes of Memory? appeared first on insideHPC.

Interview: Josh Simons on HPC Cloud Trends at SC12

Rich Brueckner — Wed, 28 Nov 2012 15:29:50 +0000

http://www.youtube.com/watch?v=XN8ZIyj35pM

In this video from SC12, Josh Simons from the VMware CTO office describes recent trends in Cloud Computing for HPC. The company is looking at how virtualization technologies could benefit supercomputing on the road to Exascale.

My message to attendees was that today’s cloud is not tomorrow’s cloud: As virtualized performance continues to advance and as cloud providers see a business value in deploying high-bandwidth, low-latency interconnects, the number of applications that will run well in a cloud environment will continue to expand. In the meantime, many single-process applications run very well in virtualized environments, MPI overheads may be acceptable for some applications, and virtualization can offer new capabilities not available in traditional, bare-metal HPC environments.

The post Interview: Josh Simons on HPC Cloud Trends at SC12 appeared first on insideHPC.

ScaleMP Steps up with Wider Platform Support plus Big Data Capabilities

Rich Brueckner — Thu, 08 Nov 2012 12:00:33 +0000

This week ScaleMP released version 5 of their vSMP Foundation software for HPC. With support for new hardware platforms such as Intel’s new Xeon Phi co-processor, this server-virtualization-for-aggregation software now enables enables large-memory applications for Big Data and Big Science. To learn more, I caught up with ScaleMP’s CEO, Shai Fultheim.

insideHPC: What’s new with the vSMP Foundation 5 release?

Shai Fultheim: There are a number of new capabilities which extend our reach and broaden our market. The most significant features are that we will support the Intel TrueScale InfiniBand as well as the Intel Xeon Phi coprocessor, virtualizing it so that it is very easy to get applications up and running on that system. In addition, we expanded processor support for the latest generation of AMD processor and the Intel E5-4600; we have added new product focused on large memory VMs (Bio informatics and big-data) as well as support for vSMP Foundation running over KVM with Ethernet using Ethernet between nodes.

insideHPC: How does vSMP Foundation work with the new Intel Xeon Phi co-processors and what advantages does this computing approach have for HPC users?

Shai Fultheim: Our work with the Intel Xeon Phi coprocessor is very different than other HPC vendors. We are providing a means that developers or end users can use to get their applications running on a system with the Intel Xeon Phi coprocessor installed. We virtualize the Intel Xeon Phi coprocessors and memory with the host processors and memory to make the system look like one large system. Then, an application can be tuned little by little to take advantage of the performance and large number of cores on the Intel Xeon Phi. In addition, we support the standard co-processor programming model for customers looking to have large-scale shared memory systems supporting significant number of Intel Xeon Phi coprocessors.

insideHPC: How does ScaleMP help users with Big Data Analytics?

Shai Fultheim: The more data that can be stored in memory the better. As the amount of data grows that needs to be analyzed, the more of that data that can be stored in easily accessible memory from all of the cores the better. Using a single address space for high speed memory will be easier and faster than using slower flash or hard disk drives. vSMP Foundation 5 allows creating virtual machines that aggregate memory of several systems, without the processors – allowing for in-memory processing using a cost optimized hardware.

insideHPC: Does it improve Hadoop performance?

Shai Fultheim: It can. By allowing the distributed nodes to be larger in the sense of more memory then more data can be stored in memory rather than on hard disk drives. Another aspect is the instead of maintaining many nodes in a cluster that are used for Hadoop, lesser nodes can be used which eases the management burden of maintaining more nodes.

insideHPC: Can vSMP Foundation co-exist with other VMs out there? How does that work?

Shai Fultheim: With vSMP Foundation 5 we will offer the ability to connect previously created VMs (like from using KVM) together to create a SMP that can be quite large. We aggregate the individual VMs and then deliver an SMP created from the smaller VMs. Our software can span individual hardware systems, giving users much larger VMs than possible before.

insideHPC:How does vSMP Foundation deliver more performance with Batched I/O?

Shai Fultheim: Processors and devices need several interactions in for each I/O operation. For example, to get a drive to pull few sectors into memory, a processor needs to access memory 2-3 times and also issue at least one I/O write operation. vSMP Foundation 5 batches those operations in a memory buffer close to the processor, to provide faster execution across the InfiniBand fabric. We have seen a 2x improvement in disk transfer rate and up to 5x improvement in the network rate in some cases. Depending on the application, your mileage may vary.

For more information, check out the ScaleMP Show Guide for SC12 and visit booth #3425.

The post ScaleMP Steps up with Wider Platform Support plus Big Data Capabilities appeared first on insideHPC.

Video: EMC – Confounding Complexity

Rich Brueckner — Wed, 10 Oct 2012 11:41:20 +0000

http://www.youtube.com/watch?v=klkDwSz0YSk

In this video, Dan Hushon, Distinguished Engineer at EMC Corporation presents: Confounding Complexity.

Dan Hushon is a Distinguished Engineer at EMC Corporation and a Sr. Director in EMC’s Office of the CTO, with specific responsibility for distributed information [Big Data] infrastructure architectures, Cloud Computing, Service Provider technologies, and the Next Generation Infrastructure supporting Big Data workloads for Cloud Scale delivery.

Recorded at the PuppetConf 2012 user group event in San Francisco. See more videos from the conference at the Puppet Labs Channel.

The post Video: EMC – Confounding Complexity appeared first on insideHPC.

Seeking Your Nominations for SVC Awards

Ralph — Wed, 03 Oct 2012 16:14:30 +0000

Nominations are now being accepted for the SVC Awards, which acknowledge excellence in Storage, Virtualization, and Cloud Computing. Judged by a panel of HPC industry editors and analysts, these awards are a great way to recognize your customers.

Nominations for the 2012 awards program are due by Oct. 17, 2012.

The post Seeking Your Nominations for SVC Awards appeared first on insideHPC.

Video: Managing HPC Clusters with MOAB

Rich Brueckner — Tue, 04 Sep 2012 12:00:24 +0000

http://www.youtube.com/watch?v=wXJU1HUVy6I

In this video, Chad Harrington from Adaptive Computing describes the coming wave of virtualization technologies in high performance computing and how the company’s Moab software helps customers manage HPC clusters of all sizes.

Recorded at VMworld 2012 in San Francisco.

The post Video: Managing HPC Clusters with MOAB appeared first on insideHPC.

Video: An Update on Virtualization Technologies for HPC

Rich Brueckner — Tue, 26 Jun 2012 23:14:03 +0000

http://www.youtube.com/watch?v=XLrQen6JRGA

In this video, Josh Simons from VMware discusses how virtualization technologies are making inroads into traditional HPC. Simons has been working on the project for two years as a representative for VMware’s CTO office.

Recorded at ISC’12 in Hamburg. You can follow progress on this effort at the Josh Simons blog.

The post Video: An Update on Virtualization Technologies for HPC appeared first on insideHPC.

Video: RDMA on vSphere – Update and Future Directions

Rich Brueckner — Mon, 02 Apr 2012 11:00:09 +0000

http://www.youtube.com/watch?v=JaFqw-VkqWo

In this video, Bhavesh Davda and Josh Simons from VMware present: RDMA on vSphere: Update and Future Directions.

Recorded at the Open Fabrics Workshop on March 26, 2012 in Monterey, CA. Slides and more are available at the conference site (login required).

The post Video: RDMA on vSphere – Update and Future Directions appeared first on insideHPC.

Video: Josh Simons from VMware on Virtualization for HPC and Big Data

Rich Brueckner — Tue, 27 Mar 2012 15:47:48 +0000

http://www.youtube.com/watch?v=E5jGn8CRutA

In this video, William Wallace from insideHPC interviews VMware’s Josh Simons on the topics of virtualization and RDMA, high performance computing, and Big Data. Recorded at the Open Fabrics Workshop on March 26, 2012 in Monterey, CA.

Josh has also posted some interesting notes on his presentation on RDMA at the Workshop:

In my introduction, I showed a few graphs from our upcoming paper, RDMA Performance in Virtual Machines using QDR InfiniBand on VMware vSphere 5 that showed we can use passthrough mode (VM DirectPath I/O) to deliver 1.75us half ping-pong latencies for Send and 3us for RDMA Read using polling completions and 7.6us using interrpt completions for RDMA Read.

The post Video: Josh Simons from VMware on Virtualization for HPC and Big Data appeared first on insideHPC.

Researchers Encouraged to Submit Proposals on Security in Virtualization

Rich Brueckner — Fri, 09 Mar 2012 16:16:33 +0000

Clipped from: labs.vmware.com (share this clip)

The good folks at VMware Labs have issued their Spring 2012 Request for Proposals on Security for Virtualized and Cloud Platforms.

Our research interests in the field of security are broad, including but not limited to the following topics: Homomorphic encryption systems and their applications in cloud environments, security isolation in mobile hypervisors, covert channels in hypervisors, multi-level security isolation guarantees with hypervisors, security implications of GPU virtualization, virtual machine introspection, anomaly detection in virtual desktop environments, leveraging virtualization to improve intrusion detection, and secure cloud computation on untrusted platforms.

Abstracts are due March 16, 2012. Winners will receive initial funding up to $150,000, so get on it! Read the Full Story.

The post Researchers Encouraged to Submit Proposals on Security in Virtualization appeared first on insideHPC.