Exascale Development Flows to Europe: Is the EU Positioned for a Leadership Role?

The enormity of the exascale development task dictates that the community leverage every last synapse of HPC genius, worldwide. This seems to be a growing consensus among HPC community thought leaders like Jack Dongarra, Thomas Sterling, and others, who have been champions of a coordinated, collaborative global approach — one that has been embodied in The International Exascale Software Project (IESP).

This initiative and its European cousins, PRACE and EESI, are designed to create a roadmap that enables the simultaneous arrival of exascale hardware and a useful software stack. 2009 and 2010 saw key meetings in France, Japan, and the UK that made progress on the roadmap concept, calling for broadly distributed development of a complete exascale software infrastructure. The IESP roadmap may play to the strengths of the EU’s development capabilities. As exascale initiatives emerge from cloistered discussion groups into the bright sunlight provided by actual funding and published job openings, it appears that many of the early checks are in the mail to HPC centers of excellence in Europe — some of which are now sporting shiny new exascale branding.

In the last six to 12 months, IBM, Intel and Cray have all made fresh investments in Europe, (see the sidebar) citing a concentration of skills and resources for the development of super-scaling applications, algorithms, and tools within the borders of the EU. Concerning this early flurry of activity, some speculate that a confluence of government policy, international investment and expanded university programs in HPC may have led to this substantial increase in exascale research activity in Europe. Others suggest that the centers and university programs have always been there and are now attracting investment because their applications provide the right test beds for exascale challenges. Another, perhaps more obvious reason may be that the HPC market in Europe has grown substantially, and companies are simply responding to a growing customer base with more collaboration.

Hard and fast explanations for this trend are not yet evident, and dissecting the scientific, academic and political drivers in order to extract a well-reasoned conclusion may take a good deal of time. In this article we’ll look at this trend and set the stage for future monitoring of its importance, staying power, and root causes.

Is this Flurry of EU Exascale investment Really Surprising?

This flurry of EU Exascale activity might surprise some, but for many who have been tracking the EU, it seems a natural progression. Across Europe, the list of major installations and influential centers is long indeed. Forschungszentrum Jülich, for example, has established a position as a leader in algorithms and system efficiency, and boasts impressive and influential HPC staff, such as Dr. Thomas Lippert. IBM and Jülich have collaborated on a number of initiatives including QPACE, a cell-processor based system featuring 26 peak teraFLOPS, which was named the world’s most energy efficient supercomputer in 2009, and the one petaFLOPS JUGENE system.

Major HPC Centers, Customers Garner Attention, Investment

Major HPC centers have certainly played a major role in the growth of the HPC market in Europe and attracted increased R&D investment from companies like Cray.

Peter J. Ungaro, CEO of Cray Inc.

Peter Ungaro, President and CEO of Cray, says, “As far as Europe is concerned, we are all in… We invest in technologies and other R&D capabilities that are important to our customers… As the European supercomputing market has become a larger part of our overall supercomputing business, it was natural that we invest more in that market and with our customers there, including investing in R&D within Europe.  Since exascale is on the agenda of many of our customers and delivering sustained exascale capability is a focus for Cray, it made sense to establish an initiative around it and partner with key European supercomputer centers, research institutions and software companies.  Today, in addition to our European Exascale Research Initiative, we have several supercomputing centers of excellence in Europe; our Cray User Group holds a number of its conferences in Europe (including this year where we just wrapped up one of our largest CUG events ever, right in Scotland); and Europe is also home to a number of our principal and senior principal engineers, the highest level engineering positions in our company.”

Brian Quinn, Director of Operations of Intel Labs Europe

To meet the exascale challenge, Intel is also expanding its laboratory complex and research programs in the EU, explains Brian Quinn, Director of Operations of Intel Labs Europe.

“The overall challenges facing exascale are broad and significant, and will require collaboration across many parties to make exaFLOPS a reality,” he says.  “These challenges will include massive processor and software parallelism, clustering, interconnect, power management, memory, resilience, and applications.  Europe can play a role in all of these areas particularly at the software and applications level… Intel has already backed this… Our engagement at the CERN open lab in Geneva (Switzerland) and our investments in the Exascale Computing Research Center in Paris, the ExaScience Lab in Leuven, Belgium and the ExaCluster Laboratory at Jülich, Germany are examples.  These labs are part of our broader Intel Labs Europe network and are closely aligned to Intel’s HPC research to reach exaFLOPS performance,” says Quinn.

A Total HPC Package: Policy, Universities and Public/Private Cooperation

That the market for the largest systems is growing in Europe is certainly a major part of the story. But the emerging trend in exascale related research extends beyond the “usual suspects” for major hardware installations. Dr. Michael Resch, Director of the HPC Center Stuttgart (HLRS), believes that a total package of policy, established centers of excellence, public/private coordination and education are responsible for the trend in EU exascale. Resch credits the EU member states with leading with their strengths.

Michael Resch, Director of the HPC Center Stuttgart (HLRS)

He comments, “The trend is driven by government reasoning: Europe — and the EU member states — have lost the hardware race. It is the software race that is open. This is certainly due to the fact that European governments can rely on a university system that is providing [excellence] to a very large group. European governments do understand that one Nobel Prize does not make up for 20 SAPs in terms of competitiveness… and economic prosperity. The long-standing existence of centers is an asset in this competition and the work of these centers has always been targeted towards general support for science and engineering and hardly ever towards any ranking in the top500. The large centers in Europe have done their homework with respect to showing their relevance for economy and society as a whole.”

With a focus on software, and academic HPC programs on the increase across Europe, the EU countries may be able to leverage what has always been a key to the American model: fund and support the university system with numerous post-graduate programs and increased research grants. The recent job ad below for a position in Ireland is an early example. It illustrates the coordination of the academic and corporate sectors, using the exascale moniker and IBM funding.

The Heterogeneous Computing Laboratory at University College Dublin (UCD), as part of the IBM Exascale activity, is seeking a suitable postdoctoral fellow and PhD candidate. These positions will be co-funded by IRCSET (Irish Research Council for Science, Engineering and Technology) and IBM through IRCSET’s Enterprise Partnership Scheme.

This job description reflects the still amorphous state of exascale. “The successful candidate will join a research team working with IBM internal and external partners towards the creation of a novel Exascale computing system. Responsibilities include working closely with the Principal Investigator to create and evaluate novel ideas.” Sounds like serious fun. Who could resist hanging out in Dublin, evaluating novel ideas? Best of all, perhaps this effort could yield a system that will finally tell us what James Joyce’s Ulysses was about.

A Long-Term Commitment and Reliable Funding

The initiatives may provide the discipline over the long haul to ensure EU leadership and well-defined roles for various centers across the Eurozone. Dr. Resch reflected on the underlying changes that have made this possible, and noted that they have been built on and informed by numerous past initiatives.

“There definitely has been a change when it comes to HPC in Europe,” he says. “We started this process back in 1998 with discussions about grid approaches that could create a European HPC sphere. Through more and more focused coordination we have gone through projects and initiatives and have now arrived at a point where we [have] created the PRACE legal entity in Brussels. Now, this in itself is significant, but what is behind PRACE is a long process of improved coordination between the large centers and a large number of research institutions. The EU has, through the framework programs, a chance to make certain topics a European priority. These are long-term activities, which give a competitive advantage compared to other regions.”

Prof. Dr. Arndt Bode, managing director of the Leibniz Supercomputing Centre (LRZ), agrees that coordinated EU efforts are well targeted and beginning to yield results in the form of reliable funding. He also credits PRACE and other initiatives.

Arndt Bode, LRZ

Says Dr. Bode, “Through PRACE and associated initiatives, the EU and its member states are pushing hard towards a coordinated and truly European approach for peta- and exascale computing. For example, in Germany the Gauss Centre for Supercomputing (GCS) with its three contributing partners, FZ Jülich, HLRS Stuttgart and LRZ Garching, is expecting a total contribution of 400 million Euros to invest in three distinct supercomputing platforms. This investment establishes the GCS as one of the principal partners of PRACE in Germany, and even with the worldwide financial situation, we expect this funding to be guaranteed. In addition, there are funding programs in Germany for peta- and exascale software, such as the recent call on software tools by the Federal Science Ministry, and the ongoing support from the Bavarian competence network for technical-scientific high performance computing (KONWIHR), or other activities such as the Munich Center of Advanced Computing (MAC) and the Munich Computational Science Centre (MCSC).”

Reliable long-term funding, an agreed upon road map, and multi-national cooperation — a very desirable formula, in theory. Without the discipline of the guiding road map, fragmentation would no doubt ensue, or at least that’s the premise behind “coordinated,” public/private programs. Pan-European officials in Brussels seem to have embraced the exascale potential through the IESP and PRACE. It remains to be seen if a bureaucratic structure will grow up around PRACE and its cousins and prove to be the glue that unifies the effort, or the sludge that slows it down. For now, the initiatives seem to be key in getting Euros flowing.

EU Software Focus to Avoid “American and Japanese Mistakes”

Dr. Resch believes the benefits of the exascale initiatives go further than the road map in that they both sustain focus, and prevent Europe from committing what he calls the Japanese and American HPC “mistakes.”

“The EU…has a federal structure that will keep us from making the “Japanese HPC Mistake.” We will not have all our eggs in a single Next Generation Supercomputer Basket but will have a variety of national approaches that will make it possible to keep many researchers involved. Furthermore, we have so little European hardware activity that we will also not make the “American HPC Mistake.” We will not make a useless hardware effort to formally achieve some performance goal ignoring the lack of adequate software to exploit that hardware.”

Brian Quinn of Intel also stresses the EU’s growing focus and role in exascale software and tools development. “We are seeing a growing competency in Europe for HPC software development building on a long tradition of good software development in Europe,” he says.

Quinn points to some key exascale-related software efforts on the part of Intel’s European labs and partners, noting that, “Our ExaScience Lab is closely aligned with Intel’s HPC research to reach exaFLOPS performance. It will focus on enabling scientific applications for exascale systems, beginning with the simulation and prediction of “space weather,” — electromagnetic activity in the space surrounding the earth’s atmosphere —, which affects both space exploration and ground-based electronics. The Lab is located at Imec and involves all the universities in Flanders. The exascale computing research Center in Paris, in partnership with Université de Versailles Saint-Quentin-en-Yvelines (UVSQ), will focus on integration of multi-petaFLOPS systems, platforms and applications optimization techniques and tools — performance enhancements at the socket and node level. The Exacluster Laboratory, in conjunction with Forschungszentrum Jülich, will conduct research to address key problems of systems management software for large heterogeneous supercomputer systems. This will include research on open exascale runtime system software, software tools, and simulation software”

The emergence of reliable funding and pan-European cooperation is echoed by Prof. Dr. Dieter Kranzlmüller of Ludwig-Maximilians-Universität München (LMU) and Member of the Board of the Leibniz Supercomputing Centre (LRZ), Germany

Dieter Kranzlmüller, LMU

“The e-infrastructure funding by the European Commission builds on the activities in each of the member states, and it is clear that throughout Europe we see a large diversity. However, the effectiveness of initiatives such as PRACE clearly provides an additional level of quality, especially for coordination across country borders. This, in turn, has opened new possibilities as can be seen by our collaboration with our French colleagues at CEA. Other examples do exist.”

Can ancient and abiding European rivalries really be so readily cast aside in the race to exascale? Can the interests of a more competitive Europe be the unifier that ensures cooperation in the coming decade of exascale development? Stay tuned.

Dr. Arndt Bode says there are plenty of leadership roles to go around: “The leadership in Europe is built on several key roles. While France is leading the European Exascale Software Initiative (EESI), Germany is leading the PRACE project series. Both projects consist of German and French partners in leading roles, and there is generally a good cooperation between individual groups from both countries. Besides that, we should mention our good cooperation with industrial partners, and both Intel and IBM have exascale labs in Jülich as well.”

Dr. Resch goes so far as to say that European players are pulling for one another to succeed. “Together with Germany, Italy, the UK, Spain, and The Netherlands, France is a driving force in Europe,” he notes. “And the Germans are happy about that. HPC in Europe has finally found a way to play as a team. And in that team, France is one leader. Together we are looking for the world championship title in HPC research with a focus on software. Companies like Intel — which increasingly draw attention to what is happening in Europe — are welcome to that team.”

A Sustainable Trend?

Will this trend be sustainable? Dieter Kranzlmüller thinks so. He told us, “We believe that this trend will be continued in the future, as supercomputing is certainly seen as an important factor for scientific advancement and breakthrough, innovation and economic development. The governments in Germany, both federal and state, have clearly expressed their understanding for this important endeavor, and the results from the scientific users on our machines support this line of thought.”

Peter Ungaro sees expanded investment in Europe as a natural evolution for Cray. Says Ungaro, “Without a doubt the European market is a big part of Cray today and, I predict, will be an even bigger part of our future.  It is very attractive and exciting for us to be increasing our activities in Europe.  We are having excellent discussions around exascale technology, new software initiatives, and how to collaborate and co-design technologies needed for the next generation of supercomputers.  At our core, Cray is an engineering company focused solely on HPC, so I think that we are an attractive company to partner with for many scientists and HPC centers in Europe. 

“Last December, we had an executive forum in Europe at which we received very positive feedback from many of the key directors of European supercomputer centers who are excited to see Cray back in a strong way across the region,” he continues.  Without a doubt, the activities, ideas and technologies that come from this exascale initiative are going to play a major role in our R&D efforts around exascale, and it is exciting for Cray to step into a larger and more active role within the European supercomputing and scientific communities.”

Does the recent investment in exascale research in the EU and the emergence of international programs like IESP reflect a “global HPC village” coming together to leverage all of its assets, or does PRACE represent a focused effort within the EU to take a leadership role in exascale to ensure competitiveness?

Global Cooperation and Competitive Advantage — an Odd Couple

Dr. Resch suggests that both may be true. “Even though we are scientists, we should be realists. We do have… very strong competition in the field of research. If you consider the global village as described by Marshall McLuhan as a reality then — yes! — HPC is part of that global village. If you considered the global village to be a world without competition, I would disagree. HPC activities are part of a worldwide competition for leadership. And, as in the real world, it is how we deal with such a competition that defines success. I see the European HPC effort as an attempt to take a leading role by providing better solutions. As a result of such a way of competing I expect that at least at the scientific level we can leverage our assets — even though the political domain might see competition to be more important than collaboration.”

Dieter Kranzlmüller agrees that the mix of competition and cooperation is an essential balance. “We believe that both aspects are true. For Europe, in order to ensure competitiveness in science and research related areas, investments in exascale computing, associated infrastructures, and human capital are a must. However, in our global world, nobody lives on [their] own, and scientific collaboration across continents is highly important for many scientific domains,” he says.

Competition and collaboration. Political and scientific. Software and hardware. Making it all come together will be the long story of the massive exascale effort. Politics may sustain exa-science in the EU for the long term through widespread distribution of funding and subprojects, seeding numbers of centers and universities across the region. Like major commercial or military aircraft projects such as Airbus in the EU or the F-22 fighter in the U.S., spreading the work over as many districts or nations as possible provides an overall resilience, and exascale development may be following that pattern in Europe.

Exascale development in Europe is visibly growing in France, Germany, Ireland and the UK. To be sure, it will be reinforced by seeding investments all over Europe —“from the Baltic to the Adriatic” to paraphrase Winston Churchill. The process may shed more light on whether or not smart politics makes smarter, faster or better science.

At this early stage, it is difficult to call out the most important drivers of this trend. Little empirical evidence can be offered to defend the notions that this is mainly the result of EU policy, or national initiatives in countries like France and Germany, or expanding excellence within the university systems of Europe, or the impact of continental initiatives like EESI and PRACE. All will be revealed in time. For now it may be enough to content ourselves with the comforting thought that “it takes a global village to raise an exascale system.”

Warm, fuzzy, and lacking any historical basis, that sentimental view may in time be replaced with the colder reality that big investment is worthwhile for Europe — worthwhile because exascale competence is a huge chip in the high stakes game of economic balance of power among the U.S., the EU, Asian powers (read China), and a re-emerging Russia. That said, the HPC planet seems flooded with good will for the moment. And that’s a rare but welcome state of affairs.