In this video from the 2013 HPC User Forum, Burak Yenier presents: The HPC Experiment – Paving the way to HPC as a Service.
For the 2nd Round of the HPC experiment, we will apply the cloud computing service model to workloads on remote Cluster Computing resources in the areas of HPC, Computer Aided Engineering, and the Life Sciences.
In related news, the HPC Experiment site has just added an online exhibit area as one-stop interactive service directory for Cloud users and service providers with focus on High Performance Computing, Big Data, Digital Manufacturing, and Computational Life Sciences.
Over at Computing Now, Art Sedighi writes that while cloud, grid, and HPC remain as distinct approaches, the twist in recent years has been the ability to coordinate and integrate these seemingly different environments. To illustrate, he describes a new peer-reviewed paper that introduces the concept of a meta-scheduler that can move workloads across all three environments.
Over the last decades, the cooperation amongst different resources that belong to various environments has been arisen as one of the most important research topic. This is mainly because of the different requirements, in terms of jobs’ preferences that have been posed by different resource providers as the most efficient way to coordinate large scale settings like grids and clouds. However, the commonality of the complexity of the architectures (e.g. in heterogeneity issues) and the targets that each paradigm aims to achieve (e.g. flexibility) remains the same. This is to efficiently orchestrate resources and user demands in a distributed computing fashion by bridging the gap among local and remote participants. At a first glance, this is directly related with the scheduling concept, which is one of the most important issues for designing a cooperative resource management system, especially in large scale settings. In addition, the term meta-computing, hence meta-scheduling, offers additional functionalities in the area of interoperable resource management because of its great proficiency to handle sudden variations and dynamic situations in user demands by bridging the gap among local and remote participants. This work presents a review on scheduling in high performance, grid and cloud computing infrastructures. We conclude by analysing most important characteristics towards inter-cooperated infrastructures.
We have invented a unique approach to building a fabric across a large number of Ethernet switches, and built a comprehensive technology platform based on this Flexible Radix Switching (FRS) technique. This innovation enables transparent integration with existing data center solutions and big improvements to networks supporting cloud, virtualization, and big data applications. These data center network solutions are superior in terms of cost, performance, robustness and ease of use.
Performance, availability and scalability requirements of large scale cloud businesses cannot be met with traditional IT approaches to storage, that typically excel in one of these areas and fall short in another,” said Charles Wuischpard, CEO Penguin Computing. “To meet the demands of our customers that require storage solutions at the petabyte scale we based our large scale storage appliance Icebreaker CS on software from Scality. With its distributed no-shared architecture and its sophisticated Advanced Resilience Configuration, Scality RING offers excellent storage scalability and great availability without compromising performance.”
The software-defined datacenter vision took the industry by storm in 2012. It represents a prescriptive model that brings the benefits of virtualization to the rest of the datacenter. Expect to see the move towards a software-defined datacenter accelerate in 2013. Networking and infrastructure security represent some of the stickiest issues when it comes to the drive to a more agile data center. And because of this strong customer interest in SDDCs, you’ll also see more networking vendors and startups modify their roadmaps to steer towards a software-defined networking strategy.
In this video from Moabcon 2013, Dick Bland and Jérôme Labat from HP present: The New Style of IT: HP Update for Moabcon 2013.
Cloud, Mobility, Security, and Big Data are transforming what the business expects from IT resulting in a “New Style of IT.” The result of alternative thinking from a proven industry leader, HP Moonshot is the world’s first software defined server that will accelerate innovation while delivering breakthrough efficiency and scale.
While the first spin of Moonshot is not targeted at HPC, Bland said that HP will be able to spin up new modules for the platform that could include FPGAs and ARM-based nodes more suited to high performance computing.
The new HPC Wales facility in Swansea has been earmarked to become south Wales’ innovation hub for businesses and academic researchers.
Both the Swansea and existing hub in Cardiff have been outfitted with the latest Fujitsu Primergy equipment with Intel Sandy Bridge processors. In this latest Phase-2 development of the pan-Wales distributed network, total capacity will grow to over 17,000 cores and nearly 320 Teraflops of processing power.
HPC Wales aims to make Welsh businesses more competitive in global markets and to grow the knowledge economy as well as creating employment opportunities,” said David Craddock, CEO of HPC Wales. Tod ate over 100 businesses have sought our advice and we have trained over 500 individuals. Since we opened for business in early 2012, demand has been particularly strong from those in the engineering, environment, life sciences and creative industry sectors.
The Swansea-based installation will boast a purpose-built datacenter with an environmentally friendly water-cooling system. Read the Full Story.
Over at Admin HPC, Dell’s Jeff Layton writes that a pair of recent use cases have helped changed his mind about the validity of using the Cloud for HPC.
At first, it was fairly easy to dismiss cloud computing for traditional HPC workloads. The “HP,” after all, stands for “high performance,” and doing anything to reduce performance is counterproductive. You are paying more and getting less. However, new workloads are being added to HPC all of the time that might be very different from the classic MPI applications in HPC and have different characteristics. The amount of computation in these new workloads is increasing at an alarming rate – so much so, that I think HPC is giving way to RC (research computing).
In this video, Marcel Vreeswijk and Hurng-Chun Lee from the NIKHEF National Institute for Subatomic Physics explain how customized grid computing workflows are key to filtering LHC datasets down to a manageable size.
The Large Hadron Collider (LHC) is the world’s largest and most complex experiment, at the cutting edge of High Energy Physics. Particle physicists use the LHC to study variations from the Standard Model and discover potential new laws of physics. The particle known as the top quark is a window to this weird and wonderful world. The LHC produces enormous amounts of data, enough to fill piles of DVDs. Without these tools, it would be impossible to pick out the collision event that could hold the clues to top quark behaviour.
Glenn K. Lockwood from SDSC writes that setting up a working set of EC2 instances that have the necessary configuration to run MPI applications can be quite daunting, so he has posted a guide.
Most guides online are kind of unhelpful in that they try to illustrate some proof of concept in how easy it is to get a fully configured cluster-in-the-cloud setup using some sort of provisioning toolchain. They gloss over the basics of exactly how to start these instances up and what to expect as far as their connectivity. Fortunately it only took me a morning to get MPI up and running, and for the benefit of anyone else who just wants to get MPI applications running on EC2 with as little effort as possible, here are my notes.
In this video, Nebula CEO Chris Kemp discusses his new product called the Nebula One and the future of cloud computing with Cory Johnson on Bloomberg Television. Kemp was formerly the CTO of NASA IT.
Nebula One brings the cloud to you, under your control, behind your firewall. It is an integrated hardware and software appliance providing distributed compute, storage, and network services in a unified system.
The Nebula One has to be cool — they’ve got Patrick Stewart and Andy Bechtolsheim in their launch video!
Over at GigaOm, David Meyer writes that European IaaS provider CloudSigma has abandoned magnetic disks for solid-state storage. After a pilot test of SolidFire’s all-SSD storage system, CloudSigma now feels confident enough to offer a service-level agreement for performance, as well as uptime.
According to CloudSigma COO Bernino Lind, the shift to SSD is a major help when it comes to handling HPC workloads, such as those of Helix Nebula users CERN, the European Space Agency (ESA) and the European Molecular Biology Laboratory (EMBL):
They want to go to opex instead of capex, but the problem is there is no-one really who does public infrastructure-as-a-service which works well enough for HPC. There is contention — variable performance on compute power and, even worse, really variable performance on IOPS [Input/Output Operations Per Second]. When you have a lot of I/O operations, then you get all over the spectrum from having a couple of hundred to having 1,000 and it just goes up and down. It means that, once you run a large big data setup, you get iowaits and your entire stack normally just stops and waits.” Lind pointed out that, while aggregated spinning-disk setups will only allow up to 10,000 IOPS, one SSD will allow 100,000-1.5 million IOPS. That mitigates that particular contention problem. “There should be a law that public IaaS shouldn’t run on magnetic disks,” he said. “The customer buys something that works sometimes and doesn’t work other times – it shouldn’t be possible to sell something that has that as a quality.”
In an April 1 blog post, Jason Stowe writes that his company Cycle Computing has launched new intelligent orchestration software called “ZeroCompute.”
ZeroCompute solves access to remote high performance cloud computing environments with a patent pending technology designed to accelerate access to HPC and BigData systems. This approach can simultaneously manage science that might normally require brontobyte datasets, and five hundred billion concurrent jobs. After years building Utility HPC & Utility Supercomputing software, Cycle’s engineers have discovered that the fastest way to execute any algorithm is to just not run it at all. Although seemingly obvious, by simply not running the science on any cores, we remove the computational challenges and data transfer bottlenecks of today’s BigCompute and BigData workloads. Because ZeroCompute “completes” the floating point in the nanosecond it takes the software to decide not to run anything, accordingly, the system exceeds a peak floating-point performance of over one billion petaflops, at a cost of $0.00 per flop.
Read the Full Story and be sure to enjoy the rest of April Fool’s Day.
You can participate in this experiment as an industrial End-User in need of instant additional computing power accessible remotely, or as a compute Resource Provider, or as a Software Provider, or as an HPC Expert.