Demo Video: One-Click HPCC Thor Cluster on AWS

 

In this video, Charles Kaminski from Lexis Nexis Risk Solutions demonstrates how easy it is to launch an HPCC Thor Cluster on AWS for Big Data analytics.

HPCC (High Performance Computing Cluster) is a cluster computing platform used to solve Big Data problems. Its unique architecture and simple yet powerful data programming language (ECL) makes it a compelling solution to solve data intensive computing needs.

HPCC is Open Source and is available for download. Try it for yourself at http://aws.hpccsystems.com.

Comment on this story

Advertisement

Accelerate x64 Applications on GPUs Using the Most Trusted HPC Compilers: The Portland Group.

Interview: T-Platforms on the Russian Road to Exascale

 

We continue our series of features on European HPC vendors with this interview of Alexey Komkov from T-Platforms, a Russian supercomputing company with global ambitions.

insideHPC: How would you describe the three main strengths of T-Platforms?

Alexey Komkov: Our company is one of the few players in the global IT market whose portfolio includes not only software and hardware solutions for clients of various categories, but also a wide range of supercomputer simulation and parallel computing services. This robust portfolio provides us with the ability to implement end-to-end HPC solutions: from design to installation, commissioning, and subsequent maintenance. In our activities, we give particular attention to collaboration and cooperation with customers of our new technologies and solutions, thus maintaining the strong spirit of innovation in the company.

insideHPC: What are some of the company’s highlights from 2011?

Alexey Komkov: Last year has seen several interesting developments that I would like to mention. “Lomonosov,” the most powerful supercomputer in Russia and CIS, took an honorable 2nd place in the Graph500 test centered on the evaluation of the effectiveness of high-performance computing systems for handling large volumes of data presented in the form of graphs. In addition, this system took 18th place in the latest edition of the Top500 global ranking of the most powerful supercomputers. Using the Lomonosov supercomputer, scholars and students of MV Lomonosov Moscow State University conduct numerous scientific and practical research programs in physics, chemistry, microbiology, and nanotechnologies, along with exploring Earth’s climatic phenomena and their relationships, and conducting special studies for the Government and many commercial companies.

Late last year we introduced a new solution, the T-Blade V-Class system, a universal foundation to build entry-level and midrange high-performance systems, as well as cloud data centers. We expect that in the near future, this system will be the main component of medium-to-high-performance supercomputer systems which are the most popular platforms in the Russian scientific community.

Also in 2011, we significantly enhanced our services portfolio to include the services for supercomputer modeling, optimization and parallelization of application software products. In addition, the construction of storage systems has become the focus for a separate line of business, and the number of our partners has increased to include such well-known leaders of the global storage market as Panasas, Net App, DDN and others.

In my opinion, perhaps the important achievement for T-Platforms is the company’s victory in the competition to design the first 10-Pflops computer system in Russia and CIS. This contest was conducted by MV Lomonosov MSU, and we, as the University’s long-standing partner, could not take this lightly. Note that we did acquire the right to design this system, and now a separate competition to select the contractor for its construction will be conducted.

We also won the competition to create the Rosnano Nanotechnology Center for the design of microelectronic components. This step will help our company to diversify its business and improve the competitiveness of the Russian microelectronics industry as we work toward developing in-house chips to be applied both within T-Platforms’ own solutions and in third-party high-tech products.

But perhaps the most significant event of the past year for us was the long-awaited entry of Vnesheconombank into our company’s capital. Cooperation with the strategic investor will allow T-Platforms to strengthen its technology leadership and raise the competitiveness of Russian supercomputer technologies in the global HPC market.

insideHPC: How is the company performing on the basis of financial results and market share?

Alexey Komkov: We strive to maintain the stability of financial flows while diversifying our profit sources. The last financial year saw increased investment in the development of new technologies, products and services. In 2010-2011, many potential customers delayed contracts for delivery of HPC-systems because of the global financial crisis, and therefore we expect significant growth in sales in 2012.

insideHPC: T-Platforms was recently named in an announcement about the development of Russian exascale technologies. Can you tell us more about this?

Alexey Komkov: Our company is part of the Russian HPC-community which is developing a comprehensive exascale strategy. As a leading Russian developer of supercomputing technologies and solutions based on these technologies, we are supported enthusiastically by the government and we are taking a direct part in the development of this strategy for different areas of science and industry in close cooperation with other players in the HPC market.

In addition, we have our own initiatives in exascale development where we are conducting the research independently.

insideHPC: What will you be showcasing at ISC’12 in Hamburg?

Alexey Komkov: We recently completed a new version of our Clustrx software solution to manage and configure computer systems. We are planning to demonstrate that, as well as a number of new hardware and software tools, at the forthcoming 2012 ISC Conference in Hamburg.

insideHPC: How important is the ISC conference to T-Platforms as the company moves to expand its business in Western Europe and beyond?

Alexey Komkov: This is a key marketing event, not only for Europe, but also throughout the global supercomputer industry. This year, it will bring together leading representatives of science, IT, and other communities and we look forward to participating.

Comment on this story

FAST Report: Flash to Run out of Gas by 2024

 

A new report from UCSD and Microsoft Research casts doubt on the future of NAND Flash technology. According the to paper, the life and performance of flash will degrade to the point that it will no longer be a viable performance solution by 2024.

In this work, we present our empirical data collected from 45 flash chips from 6 manufacturers and examine the performance trends for these raw flash devices as flash scales down in feature size. We use this analysis to predict the performance and cost characteristics of future SSDs. We show that future gains in density will come at significant drops in performance and reliability. As a result, SSD manufacturers and users will face a tough choice in trading off between cost, performance, capacity and reliability.

Download the Paper.

Comment on this story

Torsten Hoefler Wins 2012 SIAG/Supercomputing Junior Scientist Prize

 

NCSA’s Torsten Hoefler, who leads application and system performance modeling and simulation efforts for the Blue Waters project, has been selected as the recipient of the 2012 SIAG/Supercomputing Junior Scientist Prize. The award honors distinguished contributions in the field of algorithms research and development for parallel scientific and engineering computing.

Hoefler’s research revolves around performance-centric software development and deals with scalable networks, parallel programming techniques, and performance modeling. He received the award at the 2012 SIAM Parallel Processing Conference, at which he gave a presentation about his research work title Performance-oriented Parallel Programming: Integrating Hardware, Middleware and Applications.

Abstract: Parallel programming is hard, optimizing parallel programming is even harder, and writing optimal parallel programs is nearly impossible. Optimizing communication in parallel programs routinely requires dealing with low-level system details. We show portable abstractions that enable transparent optimizations but require advanced techniques in the lower layers. We conclude that scaling to larger machines demands a holistic approach to integrate hardware, middleware, and application software to develop performance-portable parallel programs.

For more information about Hoefler’s research, see www.unixer.de.

1 comment, add yours

Exascale by 2018: Crazy …or Possible?

 

By Dan Olds, Gabriel Consulting • Get more from this author

I recently saw some estimates that show we should hit exascale supercomputer performance by around 2018. That seems a bit ambitious – if not stunningly optimistic – and the search to get some perspective led me on an hours-long meander through supercomputing history, plus what I like to call “Fun With Spreadsheets.”

Right now the fastest super is Fujitsu’s K system, which pegs the Flop-O-Meter at a whopping 10.51 petaflops. Looking at my watch, I notice that we’re barely into 2012; this gives the industry another six years or so to attain 90 more petaflops worth of performance and bring us to the exascale promised land.

This implies an increase in performance of around 115% per year over the next six years. Is this possible? Let’s take a trip in the way-back machine…

Here’s a handy chart to show how long it took to move from one performance level to the next…

mega_giga_tera_peta_table

Just getting to megaflop performance took from the beginning of recorded history until 1964. If we start the clock with the Xia Dynasty at 2,000 BC, this means it took us 3,964 years to get from nothing to megaflops. This is a pretty meager rate of increase, probably somewhere around 0.17 per cent a year, but you have to factor in that everyone was busy fighting, exploring, coming up with new kinds of hats, and inventing the Morris Dance.

The first megaflop system, the Seymour Cray-designed Control Data CDC 6600, was delivered in 1964. It was a breakthrough in a number of ways: the first system to use newly-invented silicon-based processors, the first RISC-based CPU, and the first to use additional (but simpler) assist processors, called ‘peripheral processors,’ to handle I/O and feed tasks to the CPU. This was game-changing technology.

The transition from megaflop to gigaflop performance took only another 21 years with the introduction of the Cray-2, which hit the market in 1985. Seymour Cray broke away from Control Data in 1972 to start his own shop, Cray Research Inc. The Cray-2 delivered 1.9 gflops peak performance by extensively using integrated circuits (early use of modular building blocks), multiple processors (four units), and innovative full-immersion liquid cooling to handle the massive heat load. In its time, it was also game-changing technology. The Cray-2 was also highly stylish, with a futuristic design complimented by blue, red, or yellow panels. Here’s a PDF of a brochure covering the Cray-2.

Fast-forward another 11 years and we see the first system to sustain teraflop performance, the Intel-based ASCI Red system, which was also a big break from past supercomputer designs. Installed at Sandia National Lab in 1996, it’s an example of what we’ve come to expect from modern supercomputers with 9,298 Intel Pentium processors, a terabyte of RAM, and air cooling.

The compound annual performance growth rate (CAGR) for this move from gflop to tflop (another thousand-fold increase) is roughly 87.5 per cent per year, which won’t get us to exascale until midway through 2019 (just in time for the June Top500 list, I’d expect). Not too far off of the 2018 prediction, however.

Twelve years later, in 2008, the first petaflop (the IBM Roadrunner) system debuted. Achieving another 1000-fold performance increase in 12 years is equivalent to a 78 per cent compound annual growth rate. This is way faster than Moore’s Law, which has an implied CAGR of around 60%, but a little slower than the previous move from giga to teraflops. At this growth rate, we’ll reach exascale in 2020 – probably late in the year, but it might make the November 2020 Top500 list.

A mere three years after that, the K computer hit 10.51 pflops performance. The performance growth rate from Roadrunner to K? 116 per cent CAGR, which is almost exactly the growth rate necessary to deliver exascale by 2020.

Does this mean that we’ll see exascale systems in 2018 or even 2020? No, it doesn’t; it’s merely another data point in handicapping the race. This analysis simply looks at timelines; it ignores the problems inherent in housing, powering, and cooling a system that’s 1,000x faster than the current top performer, which sports more than 80,000 compute nodes, 700,000 processing cores, and uses enough power to run 12,000 households before they all get electric cars.

The technology challenges are mind-boggling, and it’s clear that simply applying ‘smaller but faster’ versions of today’s technology won’t get us over the exascale hump. It’s going to take some technology breakthroughs and new approaches. Even with these hurdles, I’m betting that we’ll see exascale performance before the end of 2020, putting us right in line with previous transitions.

But all bets are off if the Mayan prediction of global destruction in December of 2012 turns out to be true. In that case, I reserve the right to change my bet to the year 5976 – which is 2012 AD plus the 3,964 years it took us to get to megaflops. Seems like a safe enough hedge to me … ®

This article originally appeared in The Register. It appears here in its entirety as part of a cross-publishing agreement.

 

Comment on this story

Job of the Week: Software Developer at Minnesota Supercomputing Institute

 

The University of Minnesota Supercomputing Institute is seeking a Software Developer in our Job of the Week.

MSI seeks to hire a person with a programming background to join a dynamic group that provides support for researchers in scientific computation and informatics. This position will be responsible for: independently assessing needs and creating effective software solutions; identifying, troubleshooting, and resolving day-to-day application problems; accurately communicating with associated teams and users; and maintaining strong relationships with the systems staff to implement system enhancements.

Are you paying too much for your job ads? Not only do we offer ads for a fraction of what the other guys charge, our insideHPC Job Board is powered by SimplyHIred, the world’s largest job search engine.

As a reminder, we are offering FREE job listings for .EDU and .GOV domains, so email us at info @ insideHPC.com for a special discount code.

Comment on this story

PRACE Launches Advanced Training Centres

 

PRACE, the Partnership for Advanced Computing in Europe, has selected six of its member’s sites as the first PRACE Advanced Training Centres. The mission of the PRACE Advanced Training Centres (PATCs) is to carry out and coordinate training and education activities that enable the European research community to utilise the computational infrastructure available through the organisation. The long-term vision is that such centres will become the hubs and key drivers of European high-performance computing education.

The chosen sites are Barcelona Supercomputing Center, Spain; CINECA – Consorzio Interuniversitario, Italy; CSC – IT Center for Science, Finland; EPCC at the University of Edinburgh, UK; Gauss Centre for Supercomputing, Germany; and Maison de la Simulation, France. In addition to providing education and training opportunities for computational scientists in Europe, the training centres are also the main bodies responsible for producing materials for the PRACE training portal: www.prace-ri.eu/training.

Contemporary HPC systems offer unprecedented computing power and their architectures are constantly evolving. The on-going challenge has always been to up skill scientists and programmers to maximise efficiency and research output on such systems,’ said Dr Simon Wong, leader of the training work package in PRACE-2IP and Head of Education and Training at ICHEC in Ireland. ‘PRACE has shown its commitment to address this challenge by establishing the PATCs to significantly expand its training programme.”

There will be at least one PRACE PATC in operation at any one time, but the geographical locations of centres, assessed every two years, will vary over time. Training events may also be organised at locations external to PATC hosting sites.

This story originally appeared on HPC Projects. It appears here as part of a cross-publishing agreement with Scientific Computing World.

Comment on this story

Interview: ClusterVision Delivers a Total Cluster Lifecycle

 

With ISC’12 coming up in June, insideHPC is taking a closer look at European HPC vendors. This week we caught up with Dr.Tony Kent from ClusterVision to discuss how the company is leveraging it’s core strengths in high performance computing to deliver something called the Total Cluster Lifecycle.

insideHPC: How did ClusterVision get started?

Dr.Tony Kent: ClusterVision was founded approximately 10 years ago by Matthijs van Leeuwen, now CEO of Bright Computing, and Alex Ninaber, now CTO of ClusterVision. Both had a background in computational science and HPC applications, and formed ClusterVision to address the growing market need for a professional partner who had niche expertise and strong connections within the HPC industry. Still based in Amsterdam, The Netherlands, but now with offices throughout Europe, ClusterVision has since grown into a multi-million international operation, and has designed and built some of the fastest and most complex computational and database clusters in Europe, including many Top500 systems.  ClusterVision is now led by Alex Ninaber CTO, Christopher Huggins Sales & Marketing Director, and Frank van der Hout Financial Director. Matthijs van Leeuwen left ClusterVision to form Bright Computing, the developers of the cluster management suite Bright Cluster Manager, which is now included by ClusterVision as one of the critical components in our overall cluster solution.

insideHPC: Is HPC your core business?

Dr.Tony Kent: It’s more than just our core business – it’s our daily life. At ClusterVision we are 100% dedicated to HPCC (High Performance Compute Cluster) solutions. All of our customer facing staff come from technical computing or HPC application backgrounds, and we believe that our customers benefit from the undivided focus and attention to detail which such a dedication brings. Having said that, the cluster itself is not our motivation. At ClusterVision we have a saying, “It’s not the size of the cluster, it’s the size of the science” – it’s the HPC applications which our systems enable where our customer satisfaction really lies.

insideHPC: You have an impressive customer list including a number of universities in Europe. Have you also gained traction in the industrial sector as well?

Dr.Tony Kent: Yes, our early engagements focussed on the academic sector, – the early adopters who were driving the HPC technologies and applications. Universities and other research establishments are still an important market focus for ClusterVision – systems technologies advance rapidly and compute intensive applications and data requirements are becoming increasingly demanding, so we work closely with both new and existing customers in academia to ensure that their HPCC facilities keep pace. However, today ClusterVision also serves a wide range of commercial markets, from Government Laboratories, to Manufacturing, Healthcare, Exploration, Energy, Media and Finance. While the HPC sector is still seeing low double-digit growth, some of the newer commercial sectors are growing substantially faster. As we look today, some of our largest and most exciting projects are now with companies involved with applications in oil and gas exploration, automotive and aerospace engineering, and financial market analysis.

insideHPC: Your web site talks about the “total cluster lifecycle.” What do you mean by that?

Dr.Tony Kent: Total Cluster Lifecycle refers to the full design, build, and management timeline of cluster ownership. We realise that for our customers the purchase and delivery of a cluster system is just the start of their process. Customers are increasingly aware of the total ownership implications of their systems, both in terms of the operational costs such as power consumption, but also for less tangible on-going considerations like sustainability, process change requirements and skills training. Our first action is always to gain a deep understanding of our customers’ situation – not just their processing or data requirements, but their legacy systems and investments, current capabilities, applications, objectives, and of course their constraints, economically, technologically, and operationally. In this way we are able to design a solution with an optimal balance between investment and performance, but also one where informed decisions can be made upfront about the long-term ownership. Once delivered, the degree of operational management requirement varies, so we can provide anything from initial provisioning to full scale remote cluster administration. The HPCC market moves quickly, so future proofing current systems for emerging developments, and being on-hand to guide customers through upscale and enhancement projects is also an important part of our offering.

insideHPC: As a European vendor, how does ClusterVision’s offering differ from the other solutions out there in the marketplace?

Dr.Tony Kent: Being European in itself is one differentiator. Although we have customers all over the world, our core business and customer focus is in Europe, where we feel we are best able to offer an agile and personal level of service. Perhaps more important is our consideration of a complete HPCC solution – systems, software and services. High performing cluster solutions require the harmonious integration of a complex collection of hardware and software components, and the expert services which ensure that these are selected, assembled, and managed in an optimal way. Where others may focus on individual aspects of this process, we are able to offer our customers a complete, one-stop, turn-key solution. Our strong partnerships with all of the leading brand manufacturers means that our systems are never compromised in terms of component quality, while our independence ensures that we are ideally placed to provide our customers with impartial advice on technology performance and selection, as is best suited to their needs.

insideHPC: You offer support services. How critical is support to your HPC customers?

Dr.Tony Kent: Yes, support is an essential ingredient of the total cluster lifecycle concept. Most people think of support in terms of post-delivery hotline services, and repair and maintenance arrangements, and of course these are part of our standard delivery. Customers can even select different levels of support service to reflect the varying degrees of criticality of each component to their operation. However, our dedication to total cluster solutions means that we can also offer a wide range of other value-add services, throughout the cluster lifecycle. Services such as future proofed design configurations, industry standard certification, application benchmarking and optimisation, provisioning and remote administration. We also offer training to both systems administrators and end users to transfer the skills they will need to operate and utilise their cluster to its maximum capability. In addition to providing a rationalised single point of support contact, all of these services are primarily designed to ease the overall burden of cluster ownership, allowing our customers to focus on what they do best – their scientific applications.

1 comment, add yours

Why Custom HPC Processors Don’t Cut it Any More

 

Nvidia’s Sumit Gupta writes that the recent flap over the $Billion-dollar cost of the Japanese K Supercomputer brings to light that supercomputers do not have to cost nearly this much to develop and operate.

The 112 billion Yen price tag for the K computer was just the start. Add to this about 10 billion Yen ($128 million U.S.) each year to power and maintain the mammoth system, and it’s clear that the costs will really start to add up. The K system costs so much to build because the SPARC CPU at the heart of the machine is an expensive, custom-designed processor. HPC history has repeatedly shown that the development cost of custom processors is just not economically viable in the high-performance computing market.

Gupta contrasts the K supercomputer with Tsubame 2.0, a GPU-powered machine at the Tokyo Institute of Technology. He calls it “a great example of a high-performance, cost-efficient system” that currently ranks as the #5 system on the Top500 list.

Read the Full Story.

Comment on this story

Video: When Neutron Stars Collide

 

Our Video Sunday feature continues with this set of supercomputer simulations from the Max Planck Institute showing how merging neutron stars can power a short gamma-ray burst.

A spoonful of matter from a neutron star the size of a sugar cube would weigh as much as the water in all the Great Lakes.

Comment on this story

Higher Performance Through Netmap’s Smarter Packet Handling

 

Luigi Rizzo from the Università di Pisa writes that it is possible to achieve huge performance improvements in the way packet processing is done on modern operating systems.

Our experience with netmap has shown that it is possible to achieve huge performance improvements in the way operating systems employ packet processing. This result is possible without special hardware support or deep changes to existing software but, instead, by focusing on the bottlenecks in traditional packet-processing architectures and by revising design decisions in a way that could minimize changes to the system.

Read the Full Story.

Comment on this story

Video: India’s Initiative in Exascale Supercomputing

 

In this TEDx video, Dr Vijay Bhatkar presents: India’s Initiative in Exascale Supercomputing. As one of the most acclaimed and decorated scientists of India, Bhatkar is known as “The father of Indian Supercomputers.”

Comment on this story

The Schwartz is Back – This Week on inside* Publications

 

In case you missed them, here are a couple of highlights our other inside* publications this week:

  • Slidecast: SnapLogic – Simplified & Cost Effective Cloud Integration Platform. Scott Edgington from SnapLogic presents an overview of the company’s cloud integration platform. SnapLogic is the only cloud integration solution built on modern web standards and “containerized” Snaps, allowing you to easily connect any combination of Cloud, SaaS or On-premise applications and data sources.

 

Comment on this story

Whamcloud Goes Global with Lustre Training

 

Whamcloud’s Dan Ferber writes that the company has seen worldwide demand for its Lustre training, with more courses pending this March in Australia and in Alexandria, Virginia this May.

Why do we offer the training? We believe in Lustre. We think it is an incredible tool. We like teaching, and we enjoy what we learn from the many people that attend, their questions and their shared experiences. The class has no sales pitch, and no pressure to purchase Whamcloud support, proud as we are of the support we offer. The classes exist for you to learn how to install and administer Lustre.

Lustre training information is available at the Whamcloud wiki. Read the Full Story.

In related news, the Lustre community will gather in Austin for LUG 2012 on April 23-25.

Comment on this story

Chip Boffins Demo 22-nanometer Maskless Wafer-Baking

 

By Rik Myslewski in San Francisco • Get more from this author

An international consortium of chip boffins has demonstrated a maskless wafer-baking technology that they say “meets the industry requirement” for next-generation 14- and 10-nanometer process nodes.

Current chip-manufacturing lithography uses masks to guide light onto chip wafers in order to etch a chip’s features. However, as process sizes dip down to 20nm and below, doubling up on masks begins to become necessary – an expensive proposition.

At last November’s 40th birthday party for the pioneering Intel 4004 microprocesser,The Reg asked Intel Labs’ director of microprocessor technology research Shekhar Borkar if multiple masks might be the solution to ever-tinier process nodes. He told us, that, yes, it would be possible, but that adding masks increases both complexity and cost.

One solution would be to eliminate lithography masks altogether, and etch the chips directly with guided electron beams – think old-style cathode-ray tubes, but with much smaller, much more tightly controlled beams.

That’s exactly what MAPPER Lithography of Delft, The Netherlands, has done in conjunction with CEA-Leti, the French Research and Technology Institute, in a project dubbed IMAGINE, an effort joined by such industry heavyweights as TSMC andSTMicroelectronics, as well as Nissan ChemicalTOKDow ChemicalJSR Micro,SynopsysMentor GraphicsSokudoTokyo Electron, and Aselta Nanographics.

MAPPER’s breakthrough has been to demonstrate a chip-etching technology that uses 10,000 precisely directed electron beams that etch features directly onto a wafer. As explained in their announcement, “The major achievement has been obtained in resolution: 22nm dense lines and spaces and 22nm dense contact holes in positive chemically amplified resist have been successfully resolved.”

Don’t expect this technology to supplant mask-based photolithography any time soon, however. This year, MAPPER plans to introduce a “pre-production” version of its Matrix system that will have a slo-mo throughput of one wafer per hour, and which the company plans to scale up to 10 wafers per hour.

“Given the great results we have obtained at CEA-Leti thus far,” MAPPER CEO Bert Jan Kampherbeek said in his company’s announcement, “we are proud to announce that one of the first Matrix systems will be installed at CEA-Leti to enable the continuation of the IMAGINE programme.”

The Matrix system will indeed be slow – but so are current versions of one of traditional chip lithography’s waiting-in-the-wings saviors, extreme ultraviolet (EUV) lithography “To have a fab running economically, you need to build about two to three hundred wafers an hour,” the head of ARM’s Physical IP Division Simon Segars said at last year’s Hot Chips conference. “EUV machines today can do about five.”

The jury remains out on what will take over from conventional lithography when it hits the multi-mask wall – but seeing as how even EUV will require expensive masks, the maskless Matrix system and its follow-ons just became a viable contender. ®

This article originally appeared in The Register. It appears here in its entirety as part of a cross-publishing agreement.

Comment on this story

Advertisement


View All Videos

insideHPC.com is a production of insideHPC, LLC. © 2006-2011 Sitemap