If They Build It, Who Will Run It?

A long-time business colleague of mine, who we’ll call Martin for the sake of this article, called me recently to say he was packing up his family and moving to Europe. He had just accepted a position in exascale software development and agreed to a minimum two-year stay in Europe with his new company. The dynamics behind this move — and the questions it raises for the bigger picture of exascale development resources — forms the theme for this article.

There is a growing sentiment in the U.S. that we are losing ground in technology leadership and a national exascale initiative is called for. While we’ve seen some signs of commitment and funding, such as the U.S. Department of Energy’s early funding for the International Exascale Software Project, and more recently, the “Exascale Co-Design Center” solicitation, there is clearly no multi-agency collaboration at the level of what we are seeing in Europe. It does seem that the lack of a national initiative with well coordinated, collaborative organizations and significant and steady funding is placing the U.S. at a disadvantage. In fact, according to my friend Martin, “I became aware of seven exascale positions over the past month, and six of them were in Europe.”

What’s going on here: is HPC leadership quietly slipping out of the United States?

PRACE sets the pace

Just three months ago, the representatives of 19 nations came together to establish PRACE, the Partnership for Advanced Computing in Europe. PRACE has already assembled a commitment for a staggering level of investment targeted to leadership class systems in the multi-petaFLOPS range — with a clear eye on exascale. Italy, France, Germany and Spain have each committed €100 million. The EU has already invested €10 million for an early project phase and €20 million for the first implementation project. Add on to this their commitment for another €20 million each for the 2nd and 3rd implementation projects, and this amounts to an investment commitment of €470 million over five (5) years. An additional in-kind funding agreement by the partners in the EU projects accounts for another €38 million that’s in the pipeline. This explains why some of the public information refers to an investment pool of €500 million.

And it’s not just the money. The users of these extreme scale systems will have an entire support network available to them — including experts in porting, tuning and scaling applications. PRACE even offers a comprehensive training program designed to help scientists, researchers and students take full advantage of these new systems.

This Research Infrastructure, referred to as the PRACE RI, is actually an international non-profit association with its seat in Brussels. As of this writing, there were 19 different countries participating.

But, as we’ve heard so many times before, getting to exascale is a far bigger effort than just building the systems. PRACE seems to be keenly aware of this, with their attention to training and support. As we cross the mid-decade line, we are likely to see several hundred — if not several thousand — resources involved at some level of exascale R&D. Where will these folks come from? And just because the EU has taken an aggressive funding stance, does that necessarily mean Europe will lead with exascale development?

Dr. Kimmo Koski, Managing Director of CSC, the Finnish IT Center for Science and the Vice Chair of the PRACE Council, doesn’t think so. “Since most of the supercomputing manufacturing is in the USA and Japan, and I doubt it will change in ten years, they will probably install the first systems somewhere close. It’s not so important where the top hardware is located. The most important factor is who can best benefit from it and make real scientific breakthroughs. For that competition, Europe has a good position — regardless of where the first systems will be.”

Koski added another pearl of wisdom. “Actually, it’s not the computer that solves problems, but those who are competent enough to use them.”

Yes, my point exactly. If they build it, who will run it?

And, if they fund it, who will build it? Who will architect it? Who will write the applications?

The money is on the table — let the hiring begin

So going back to Martin, who was aware of seven exascale positions last month, it got me wondering. Which companies are actually starting to look for exascale talent. And, are the majority of those early positions going to be in Europe?

We spoke with HPC technology recruiter Penelope Clayton-Smith. Clayton-Smith has been dedicated to HPC technology recruitment for 30 years and is a well known and highly respected resource in the HPC community. She started as a recruiter for Control Data Corporation, Seymour Cray’s first company. Today, the company she owns, Quantum Recruitment Inc. is actively involved in recruiting selected talent to fill a number of exascale R&D positions for clients.

According to Clayton-Smith, there may be a logical reason why we’re seeing more activity in Europe. “It is somewhat easier because the immigration issues are handled differently in Europe. To fill those positions, I can draw talent from anywhere in the world.”

She also has some concerns about the talent pool for exascale positions within the U.S. “We are at an interesting crossroads for HPC in the U.S. We have both vertical and horizontal growth in the HPC market. As we push toward faster and more complex machines on the extreme cutting edge, we are also seeing HPC being flattened into new markets. I have grave concerns that we (the U.S.) do not have the talent pool necessary to fill these positions.”

So, I’ll go out on a limb and guess that for those of you who care about U.S. technology leadership, losing members of this very limited talent pool to Europe is not a good thing.

Clayton-Smith believes the trend will change and may already be occurring. She currently has several active searches for computer scientists to fill exascale R&D positions in the U.S. And although she wouldn’t comment on what company these positions are for, my guess would be IBM or Cray. But she is keenly aware that for now she is drawing from a very small pool of candidates. “Although sales and marketing positions in HPC have taken a serious hit, the same is not true for technical personnel. The demand remains high for these types of individuals.”

Intel is one company that clearly has stepped up its game and is actively staffing for exascale positions, but from what we’ve seen so far those are all in Europe. We’ll cover this more in our feature, in-depth interview with Richard Dracott, acting director of Intel’s exascale labs in Europe.

According to Clayton-Smith, early discussions around exascale-related positions will likely be spread across government, academia and industry because of the strong collaborative nature of exascale development. But again, she cautions that now is the time for action. “We need to continue to encourage STEM education and employment among our own citizenry and stop allowing the dilution of our already limited talent pool.”

Will the talent pool ultimately determine who is first with an exascale system? Koski doesn’t see it as a race for fielding the first exascale system. According to Koski, “Being first with hardware power is obsolete unless you have the software and competence to utilize the systems and achieve real results. An exascale computer is useless or at least not much different from a petascale system without suitable code that scales. This is a major effort of development, which requires much more than just hardware. Much of these scaling efforts have global participation so multiple countries will collaborate.”

Koski’s point makes sense. Does it even matter where the jobs are if everyone is going to collaborate? He continues, “I think more collaboration towards joint exascale systems is a must since it is much more cost-efficient to share the most expensive high-end resources.”

So, who are the ideal candidates for those first few jobs that Clayton-Smith is trying to fill? She responds, “I look for a Ph.D. in computational science who can wrap their thoughts around very large problems; someone who wants to be challenged on a daily basis, and can defend their ideas, while playing well with others. Given that the applications have the potential to scale to billions of threads, the ability to think in complex software environments is vital.”

Pack your bags son — we’re moving to Europe

Global sharing is a wonderful idea. And we are already seeing it at some level. But this is HPC. This is a highly charged, extremely competitive community and there should be no mistake that when we move into that final stretch of seeing the first exascale systems coming to life, global cooperation will be squashed by national pride. If the first exascale system is fielded in Europe, because that’s where the talent is, does it matter if much of it was built in the U.S.? Of course not. The gold medal for technology leadership pride will go to Europe.

Today, maybe. Tomorrow, not likely.

If things don’t change in the U.S., some very smart people seem to be in agreement that we’ll likely see just that — the bragging rights for the first exascale deployment going to Europe — or China.

Since China seems to be taking this on without the need for imported resources, we have no idea what they will enter in the race — but they are a very serious contender. It looks like the EU is putting a greyhound in the race against the U.S. dachshund. Clearly in the U.S., the Department of Energy is setting the pace for exascale R&D and, with the International Exascale Software Project, has laid an important foundation for building a global R&D community. But many believe that just isn’t enough. Without a national exascale initiative and strong multi-agency collaborative goals, U.S. technology leadership will continue to slip away.

Clayton-Smith points to one early step the U.S. HPC community could take. “We need more implementation of collaborative work environments that are geographically independent, and we need the employers, government agencies, private sector and universities to heartily embrace those environments.”

In closing, a reality check from Kimmo Koski: “It is wise to remember that exascale is expected to be available some time in 2018-2019. At the moment, there is not much code running in petascale, yet, so some pure realism is probably useful.”