insideHPC » Cloud HPC

A Review of Meta-schedulers for HPC, Grid and Cloud

Rich Brueckner — Sat, 04 May 2013 15:08:10 +0000

Over at Computing Now, Art Sedighi writes that while cloud, grid, and HPC remain as distinct approaches, the twist in recent years has been the ability to coordinate and integrate these seemingly different environments. To illustrate, he describes a new peer-reviewed paper that introduces the concept of a meta-scheduler that can move workloads across all three environments.

Over the last decades, the cooperation amongst different resources that belong to various environments has been arisen as one of the most important research topic. This is mainly because of the different requirements, in terms of jobs’ preferences that have been posed by different resource providers as the most efficient way to coordinate large scale settings like grids and clouds. However, the commonality of the complexity of the architectures (e.g. in heterogeneity issues) and the targets that each paradigm aims to achieve (e.g. flexibility) remains the same. This is to efficiently orchestrate resources and user demands in a distributed computing fashion by bridging the gap among local and remote participants. At a first glance, this is directly related with the scheduling concept, which is one of the most important issues for designing a cooperative resource management system, especially in large scale settings. In addition, the term meta-computing, hence meta-scheduling, offers additional functionalities in the area of interoperable resource management because of its great proficiency to handle sudden variations and dynamic situations in user demands by bridging the gap among local and remote participants. This work presents a review on scheduling in high performance, grid and cloud computing infrastructures. We conclude by analysing most important characteristics towards inter-cooperated infrastructures.

Read the Full Story or Download the paper (PDF).

The post A Review of Meta-schedulers for HPC, Grid and Cloud appeared first on insideHPC.

Video: Infinetics – Innovative Topologies

Rich Brueckner — Fri, 03 May 2013 19:52:44 +0000

In this video from the 2013 Open Fabrics Developer Workshop, Harry Quackenboss and Ratko Tomic from Infinetics present: Innovative Topologies.

We have invented a unique approach to building a fabric across a large number of Ethernet switches, and built a comprehensive technology platform based on this Flexible Radix Switching (FRS) technique. This innovation enables transparent integration with existing data center solutions and big improvements to networks supporting cloud, virtualization, and big data applications. These data center network solutions are superior in terms of cost, performance, robustness and ease of use.

Download the slides (PDF). You can check out more OFA videos at our Open Fabrics Workshop Video Gallery.

The post Video: Infinetics – Innovative Topologies appeared first on insideHPC.

Penguin Computing Unveils Large-Scale Storage Platform

Staff — Tue, 30 Apr 2013 13:27:41 +0000

Penguin Computing has revealed its new Cloud CS Storage Platform that will utilize Scality’s RING Organic Storage software.

Performance, availability and scalability requirements of large scale cloud businesses cannot be met with traditional IT approaches to storage, that typically excel in one of these areas and fall short in another,” said Charles Wuischpard, CEO Penguin Computing. “To meet the demands of our customers that require storage solutions at the petabyte scale we based our large scale storage appliance Icebreaker CS on software from Scality. With its distributed no-shared architecture and its sophisticated Advanced Resilience Configuration, Scality RING offers excellent storage scalability and great availability without compromising performance.”

Read the Full Story.

The post Penguin Computing Unveils Large-Scale Storage Platform appeared first on insideHPC.

Video: The Sofware-defined Datacenter & RDMA in Virtual Environments

Rich Brueckner — Sun, 28 Apr 2013 11:00:21 +0000

In this video from the 2013 Open Fabrics Developer Workshop, VMware’s Josh Simons presents: The Sofware-defined Datacenter.

The software-defined datacenter vision took the industry by storm in 2012. It represents a prescriptive model that brings the benefits of virtualization to the rest of the datacenter. Expect to see the move towards a software-defined datacenter accelerate in 2013. Networking and infrastructure security represent some of the stickiest issues when it comes to the drive to a more agile data center. And because of this strong customer interest in SDDCs, you’ll also see more networking vendors and startups modify their roadmaps to steer towards a software-defined networking strategy.

Read the Full Story or Download the Slides (PDF).

In this follow-up presentation, Bhavesh Davda from VMware gives a talk on RDMA in Virtual Environments.

The post Video: The Sofware-defined Datacenter & RDMA in Virtual Environments appeared first on insideHPC.

Video: Lustre on Amazon Web Services

Rich Brueckner — Wed, 24 Apr 2013 17:13:07 +0000

In this video from the Lustre User Group 2013 conference, Robert Read from Intel presents: Lustre on Amazon Web Services.

Download the slides (PDF). You can check out more Lustre presentations at our LUG 2013 Video Gallery.

The post Video: Lustre on Amazon Web Services appeared first on insideHPC.

Video: The New Style of IT – HP Moonshot Update for Moabcon 2013

Rich Brueckner — Fri, 19 Apr 2013 18:39:11 +0000

In this video from Moabcon 2013, Dick Bland and Jérôme Labat from HP present: The New Style of IT: HP Update for Moabcon 2013.

Cloud, Mobility, Security, and Big Data are transforming what the business expects from IT resulting in a “New Style of IT.” The result of alternative thinking from a proven industry leader, HP Moonshot is the world’s first software defined server that will accelerate innovation while delivering breakthrough efficiency and scale.

While the first spin of Moonshot is not targeted at HPC, Bland said that HP will be able to spin up new modules for the platform that could include FPGAs and ARM-based nodes more suited to high performance computing.

View the Slides on Slideshare.

The post Video: The New Style of IT – HP Moonshot Update for Moabcon 2013 appeared first on insideHPC.

New Supercomputing Hub Launches in Swansea

Rich Brueckner — Wed, 17 Apr 2013 15:14:08 +0000

The new HPC Wales facility in Swansea has been earmarked to become south Wales’ innovation hub for businesses and academic researchers.

Both the Swansea and existing hub in Cardiff have been outfitted with the latest Fujitsu Primergy equipment with Intel Sandy Bridge processors. In this latest Phase-2 development of the pan-Wales distributed network, total capacity will grow to over 17,000 cores and nearly 320 Teraflops of processing power.

HPC Wales aims to make Welsh businesses more competitive in global markets and to grow the knowledge economy as well as creating employment opportunities,” said David Craddock, CEO of HPC Wales. Tod ate over 100 businesses have sought our advice and we have trained over 500 individuals. Since we opened for business in early 2012, demand has been particularly strong from those in the engineering, environment, life sciences and creative industry sectors.

The Swansea-based installation will boast a purpose-built datacenter with an environmentally friendly water-cooling system. Read the Full Story.

The post New Supercomputing Hub Launches in Swansea appeared first on insideHPC.

Jeff Layton on the Cloud’s Changing Role in HPC

Rich Brueckner — Sun, 14 Apr 2013 11:00:32 +0000

Over at Admin HPC, Dell’s Jeff Layton writes that a pair of recent use cases have helped changed his mind about the validity of using the Cloud for HPC.

At first, it was fairly easy to dismiss cloud computing for traditional HPC workloads. The “HP,” after all, stands for “high performance,” and doing anything to reduce performance is counterproductive. You are paying more and getting less. However, new workloads are being added to HPC all of the time that might be very different from the classic MPI applications in HPC and have different characteristics. The amount of computation in these new workloads is increasing at an alarming rate – so much so, that I think HPC is giving way to RC (research computing).

Read the Full Story.

The post Jeff Layton on the Cloud’s Changing Role in HPC appeared first on insideHPC.

Video: Tales from the Grid – In Search of the Top Quark

Rich Brueckner — Fri, 12 Apr 2013 15:16:44 +0000

In this video, Marcel Vreeswijk and Hurng-Chun Lee from the NIKHEF National Institute for Subatomic Physics explain how customized grid computing workflows are key to filtering LHC datasets down to a manageable size.

The Large Hadron Collider (LHC) is the world’s largest and most complex experiment, at the cutting edge of High Energy Physics. Particle physicists use the LHC to study variations from the Standard Model and discover potential new laws of physics. The particle known as the top quark is a window to this weird and wonderful world. The LHC produces enormous amounts of data, enough to fill piles of DVDs. Without these tools, it would be impossible to pick out the collision event that could hold the clues to top quark behaviour.

Check out more Tales from the Grid on YouTube.

The post Video: Tales from the Grid – In Search of the Top Quark appeared first on insideHPC.

How to Set Up an MPI Cluster on Amazon EC2

Ralph — Sun, 07 Apr 2013 15:39:31 +0000

Glenn K. Lockwood from SDSC writes that setting up a working set of EC2 instances that have the necessary configuration to run MPI applications can be quite daunting, so he has posted a guide.

Most guides online are kind of unhelpful in that they try to illustrate some proof of concept in how easy it is to get a fully configured cluster-in-the-cloud setup using some sort of provisioning toolchain. They gloss over the basics of exactly how to start these instances up and what to expect as far as their connectivity. Fortunately it only took me a morning to get MPI up and running, and for the benefit of anyone else who just wants to get MPI applications running on EC2 with as little effort as possible, here are my notes.

Read the Full Story.

The post How to Set Up an MPI Cluster on Amazon EC2 appeared first on insideHPC.

Video: Nebula One Brings the Cloud to You

Rich Brueckner — Fri, 05 Apr 2013 10:00:30 +0000

In this video, Nebula CEO Chris Kemp discusses his new product called the Nebula One and the future of cloud computing with Cory Johnson on Bloomberg Television. Kemp was formerly the CTO of NASA IT.

Nebula One brings the cloud to you, under your control, behind your firewall. It is an integrated hardware and software appliance providing distributed compute, storage, and network services in a unified system.

The Nebula One has to be cool — they’ve got Patrick Stewart and Andy Bechtolsheim in their launch video!

The post Video: Nebula One Brings the Cloud to You appeared first on insideHPC.

CloudSigma Consolidates on SSDs for HPC IaaS

Rich Brueckner — Wed, 03 Apr 2013 07:14:53 +0000

Over at GigaOm, David Meyer writes that European IaaS provider CloudSigma has abandoned magnetic disks for solid-state storage. After a pilot test of SolidFire’s all-SSD storage system, CloudSigma now feels confident enough to offer a service-level agreement for performance, as well as uptime.

According to CloudSigma COO Bernino Lind, the shift to SSD is a major help when it comes to handling HPC workloads, such as those of Helix Nebula users CERN, the European Space Agency (ESA) and the European Molecular Biology Laboratory (EMBL):

They want to go to opex instead of capex, but the problem is there is no-one really who does public infrastructure-as-a-service which works well enough for HPC. There is contention — variable performance on compute power and, even worse, really variable performance on IOPS [Input/Output Operations Per Second]. When you have a lot of I/O operations, then you get all over the spectrum from having a couple of hundred to having 1,000 and it just goes up and down. It means that, once you run a large big data setup, you get iowaits and your entire stack normally just stops and waits.” Lind pointed out that, while aggregated spinning-disk setups will only allow up to 10,000 IOPS, one SSD will allow 100,000-1.5 million IOPS. That mitigates that particular contention problem. “There should be a law that public IaaS shouldn’t run on magnetic disks,” he said. “The customer buys something that works sometimes and doesn’t work other times – it shouldn’t be possible to sell something that has that as a quality.”

Read the Full Story.

The post CloudSigma Consolidates on SSDs for HPC IaaS appeared first on insideHPC.

New ZeroCompute Utility Punts the Brontobytes

Rich Brueckner — Mon, 01 Apr 2013 15:57:58 +0000

In an April 1 blog post, Jason Stowe writes that his company Cycle Computing has launched new intelligent orchestration software called “ZeroCompute.”

ZeroCompute solves access to remote high performance cloud computing environments with a patent pending technology designed to accelerate access to HPC and BigData systems. This approach can simultaneously manage science that might normally require brontobyte datasets, and five hundred billion concurrent jobs. After years building Utility HPC & Utility Supercomputing software, Cycle’s engineers have discovered that the fastest way to execute any algorithm is to just not run it at all. Although seemingly obvious, by simply not running the science on any cores, we remove the computational challenges and data transfer bottlenecks of today’s BigCompute and BigData workloads. Because ZeroCompute “completes” the floating point in the nanosecond it takes the software to decide not to run anything, accordingly, the system exceeds a peak floating-point performance of over one billion petaflops, at a cost of $0.00 per flop.

Read the Full Story and be sure to enjoy the rest of April Fool’s Day.

The post New ZeroCompute Utility Punts the Brontobytes appeared first on insideHPC.

Video: The UberCloud HPC Experiment – Paving the Way to HPC as a Service

Rich Brueckner — Sat, 16 Mar 2013 11:00:29 +0000

In this video from the HPC Advisory Council Switzerland Conference, Wolfgang Gentzsch presents: The UberCloud HPC Experiment – Paving the Way to HPC as a Service.

You can participate in this experiment as an industrial End-User in need of instant additional computing power accessible remotely, or as a compute Resource Provider, or as a Software Provider, or as an HPC Expert.

Read the Full Story or Download the Slides (PDF).

The post Video: The UberCloud HPC Experiment – Paving the Way to HPC as a Service appeared first on insideHPC.

PRACE Success Story Wins High Praise

Rich Brueckner — Mon, 11 Mar 2013 17:19:57 +0000

Since 2012, PRACE has been offering European companies an industrial R&D service, based on a set of complementary high-level services including information and networking, training, access to leading HPC resources and expertise, and code-enabling of open-source applications. Now this documented project offer has been recognised as an effort in catalysing European industrial competitiveness.

The FP7 Success Story Competition highlights the three best success stories from the FP7 Capacities funding programme in e-Infrastructures. Project success stories in the competitive industry category show how the project made Europe a more attractive location to invest in research and innovation, by promoting activities where businesses set the agenda. The project is aimed at helping innovative SMEs to grow into world-leading companies.

PRACE’s proposal paper, How E-research Infrastructures can Catalyse European industrial Competitiveness, was selected as the winner of the competitive industry category. The paper describes PRACE’s continuous commitment to develop services for industry to benefit from PRACE research infrastructure.

Since the establishment of the PRACE Open R&D industrial offer in January 2012, PRACE has been able to attract more than 10 European companies; large companies as well as SMEs (Small and Medium Enterprises) for using its HPC facilities as well as the other high-value services.

This award in ‘competitive industries’ will foster our motivation to work on engaging industrial users on the PRACE research infrastructure in order to boost European competitiveness,” said Stephane Requena, author of the project paper and member of the board of directors of PRACE. “PRACE is working on increasing the use of a leading European infrastructure by all academic and industrial communities and is catalysing technological transfer between academia and industry through open innovation projects. In that sense we are working in the field of the FP7 funded PRACE-3IP implementation project on a tailored evangelisation programme called SHAPE (SME HPC Adoption Programme in Europe) which aims to help SMEs to co-design and demonstrate a concrete industrial project on PRACE facilities.”

Download the award-winning success story (PDF).

This story appears here as part of a cross-publishing agreement with Scientific Computing World.

The post PRACE Success Story Wins High Praise appeared first on insideHPC.

Video: QMachine – Commodity Supercomputing with Web Browsers

Rich Brueckner — Sat, 09 Mar 2013 16:53:40 +0000

In this video, Sean Wilkinson from the University of Alabama at Birmingham demonstrates QMachine, a web service that allows ordinary web browsers to execute distributed workloads, all without installing anything.

QMachine (QM) is a web service that uses Quanah to create a distributed computer that can use ordinary web browsers as ephemeral nodes. It contains three main components: an API server, a web server, and a website. The API server and the web server are both implemented in Node.js and available for use in server environments via NPM. The API server supports CORS and configurable persistent storage for a variety of popular databases, including Apache CouchDB, MongoDB, PostgreSQL, Redis, and SQLite.

Read the Full Story.

The post Video: QMachine – Commodity Supercomputing with Web Browsers appeared first on insideHPC.

Video: Grid Computing Powers Research for Better Anesthetics

Rich Brueckner — Thu, 07 Mar 2013 16:54:31 +0000

In this video, researchers from Utrecht University use grid computing to digitally modify molecules found in cone snail venom in an effort to develop new anesthetics. The grid allows them to run a lot of trial and error tests extremely quickly to look for the right molecular shape that will be the perfect fit for the pain receptors in humans.

Read the Full Story.

The post Video: Grid Computing Powers Research for Better Anesthetics appeared first on insideHPC.

Video: Enterprise HPC in the Cloud – Fortune 500 Use Cases

Rich Brueckner — Tue, 05 Mar 2013 18:39:39 +0000

Over at the Compute Cycles Blog, Jason Stowe from Cycle Computing writes that the recent AWS: reInvent conference featured a series of talks on Enterprise Use Cases of HPC in the Cloud. In this video, Stowe introduces Cloud HPC talks from Johnson & Johnson, Pacific Life, Hartford Insurance, and Life Technologies.

Read the Full Story.

The post Video: Enterprise HPC in the Cloud – Fortune 500 Use Cases appeared first on insideHPC.

Grid Simulation Plays the Epigonion for the First Time Since 500 B.C.

Rich Brueckner — Thu, 28 Feb 2013 16:17:31 +0000

In this video, researchers use the power of the European Grid Infrastructure to simulate the sounds of the Epigonion, a Greek musical instrument that hasn’t been heard for centuries.

Using a technique called physical modelling, Domenico Vincinanza recreated the sound of the instrument’s 48 strings as digital files. With the help of grid computing resources from the European Grid Infrastructure, it took him just a few hours. In a single core computer he would need a month. The epigonion’s sounds can now be downloaded and played by any musician using a simple keyboard.

Read the Full Story.

As you may recall, Vicinanza performed live from the insideHPC booth at SC11, turning the conference twitter stream into a concert of electronic music. Using an original Java application called Sonify running on the European Grid Infrastructure using GEANT, Internet2 and SCinet, the pulse of the SC11 communiverse was brought to life.

The post Grid Simulation Plays the Epigonion for the First Time Since 500 B.C. appeared first on insideHPC.

Interview: Wolfgang Gentzsch on Cloud Sustainability

Ralph — Wed, 27 Feb 2013 16:40:26 +0000

Over at International Science Grid This Week, Andrew Purcell catches up with Wolfgang Gentzsch to learn more about the challenges of cloud sustainability in Europe.

Sustainability in the context of cloud infrastructures means the long-term maintenance of their services for a wider user and customer community. Researchers who previously have used their own limited resources are benefiting tremendously from clouds: for example, by avoiding long waiting times for their compute jobs; enjoying on-demand availability of resources; experiencing no peak-demand bottlenecks; having the best-suited resources always available at their fingertips; not having to deal with cumbersome procurements; and having no need to maintain their own expensive IT infrastructure. Clouds serve larger communities and, therefore, sustainability is key.

Gentzsch is presenting on the Ubercloud HPC Experiment this week at the CloudScapeV Conference in Brussels. Read the Full Story.

The post Interview: Wolfgang Gentzsch on Cloud Sustainability appeared first on insideHPC.