From Capitol Hill to Silicon Valley to the Great Wall of China

Exascale Crusaders Prepare for Battle

DOE Needs Battlefield Commander

June 2013
Download this article as a PDF * For related stories, visit The Exascale Report Archives.

“It’s not just that exascale is so much harder, it’s the fact that we don’t have someone in a high-level leadership position at DOE willing to go to battle with OSTP and OMB to push the exascale agenda.”

ALL EYES TURN TO CHINA. AGAIN.
By now, everyone has heard the news: China owns the fastest supercomputer in the world. Again.

In 2010, the first time the Chinese moved to the Number One position on the TOP 500 list of the most powerful l computer systems in the world, it was easy for skeptics to say it was a fluke. It was a purpose-built system intended to win a benchmark, we were told. Now those same critics have gone silent – they have been embarrassed by the U.S. once again being displaced by Chinese ingenuity.
The second time for the Chinese to take this top position is a far more important milestone and a statement of determination. The system is not based solely on Intel technology; some articles would have you believe – wondering whether this is really a Chinese system, or essentially a U.S. system built in another country. No, this is a Chinese system – that happens to use some core technology from Intel combined with a lot of other homegrown technology, demonstrating a remarkable Chinese R&D capability and computing leadership.

According to the research and technology development company, Battelle, China is expected to exceed the U.S. in R&D spending in about 10 years. While that is overall R&D, when it comes to very high end, advanced computing R&D, China seems to already be leaving the U.S. in the dust.

And, according to a McKinsey Quarterly article by Gordon Orr and Erik Roth, Beijing Genomics Institute (BGI) is the world’s largest genetic-sequencing company, very likely sequencing more genetic material than Harvard University and the Massachusetts Institute of Technology combined, while developing some of the world’s most advanced biologic-computing models. That, dear readers, is the environment and the application in which exascale systems will change the world. The Chinese government clearly understands the potential economic and social impact of extreme scale computing, and has the political infrastructure and will to allocate the necessary funds to own this space.

Flashback: 2010
Three years ago, The Exascale Report raised the following question: “Will politicians over the next 10 years help to drive the [exascale] effort or get in the way? Will China, Japan, South Korea, and Russia participate in this global effort for the long haul with coordinated programs and government funding, or move forward on their own?”

At that time, a number of spokespeople from the HPC community went on record saying we shouldn’t look at the challenge of achieving exascale-level computation as a competition or as a race. The widespread belief was that most nations would work together – collaborate – and we would all get there together. As it turns out, that was wishful thinking.

Collaboration has quickly turned into fierce competition. The exascale race track is a battlefield. And while many nations are putting their troops on the field with new uniforms and modern weapons, as it were, the U.S. labs, our front line of defense, are trying to cobble together the necessary funds to keep their troops from giving up. Shabbily clothed, poorly armed, and without a champion to carry their ensign into battle, U.S. HPC crusaders appear to be overmatched.

The difficult economic and sequestration climate of the past several years has created an ‘every man for himself’ attitude in Washington, and the nation’s leading technology businesses, with no long-term commitment from the government have no choice but to focus on product strategies and revenue streams that will keep them afloat. The high-risk, high-reward exploration necessary to uncover breakthroughs in technology and new approaches simply can’t be funded by commercial entities alone. Our government must take a leadership role.

“The sad situation today is that corporate economic interests are driving political decisions globally and much-needed constructive discussions of long-term strategy and the associated longer-term funding such as an ongoing exascale initiative, get pushed to the back burner because they don’t offer current administrators short-term results they can brag about, and because the U.S. has no high level champion fighting for the exascale cause. This coupled with congressional budget constraints means the U.S. has little hope for securing adequate funding for longer-term research programs that will outlast changing political party agendas.” *anon

WHO WILL STEP UP AS THE U.S. BATTLEFIELD COMMANDER?
“Even if the new DOE leadership gives exascale the evangelism it needs, someone still needs to get OSTP on board. This nation needs OSTP to be a champion for exascale development. DOE needs a leader who is both willing and capable of building a strong support network, and that means getting OSTP and OMB on board, if we are to regain a global HPC leadership position. Every aspect of the U.S. economy depends on this.” *anon

There is one person who could lead the fight for the U.S. to regain lost ground in extreme computing technology leadership, the newly appointed Secretary of Energy, Ernest Moniz. It appears he has a number of willing and able soldiers – but they need him to lead the charge. Captains of the community such as the Acting Director of the Advanced Scientific Computing Research program (ASCR), Barbara Helland, and the head of Research for ASCR, William Harrod, can only do so much without adequate budgets and strong, trusted leadership. This is a battle that Moniz needs to lead.

Leadership – that is, the lack of leadership – gets to the heart of the question:
Why has China once again taken the HPC technology lead with bragging rights to the world’s most powerful computer?

Why is it that even with strong bi-partisan support for advanced HPC computing research, the U.S. is still struggling to get adequate exascale funding to keep pace with China, Japan, Russia, India, and the EU?

The U.S.’s ability to achieve and maintain a strong HPC technology leadership position in the past was possible because of the leadership of people like Vic Reis, Gil Weigand, and Paul Messina.

These three names are always mentioned warmly – and with great respect. Each of them was willing to take the fight to Capitol Hill and not back down. They fought hard for HPC budget appropriations – and the U.S. benefited tremendously from their efforts.

The following are some of the most appropriate comments from the HPC community weighing in on this topic:
“Steven Chu was too much of a scientist. Fighting wasn’t in his DNA.” *anon

“We’ve faced technology transitions and what some have called paradigm shifts before – and did those with rather impressive results. The big difference this time around is the leadership and their attitude(s), or lack thereof, toward stepping on toes and fighting for a longer-term U.S. exascale initiative with funding that will keep the U.S. competitive.” *anon

“It’s not just that exascale is so much harder, it’s the fact that we don’t have someone in a high-level leadership position at DOE willing to go to battle with OSTP and OMB to push the exascale agenda.” *anon

“These are very strange times indeed. Just weeks after several of our HPC community leaders testified before Congress to the critical importance of advanced computing leadership, specifically exascale, to economic interests and national defense, we all learn that China has again claimed ownership of the world’s most powerful computer. That achievement did not happen overnight, and one would think this would have inspired legislators to move far more aggressively. I don’t understand how any of them can pretend to be surprised by this development. Shame on Congress and shame on OSTP and DOE for letting this happen.” *anon

“The global quest to reach exascale-level computation has not only spawned a race of epic proportions, it has evolved into a geo-political battlefront where the weapons are technology innovation, human resources, and big fat checkbooks. You need all three to win. If you pick only two, you can’t possibly win this battle.” *anon

Capitol Hill Sheep Return From Pasture
There is a rumble on the streets and more than a few whispered comments around the water coolers in the Washington, D.C. Congressional office buildings. “How did this happen?” “Why is a Chinese supercomputer being built with Intel microprocessors?” “Isn’t this more of a U.S. system than a Chinese system?” “Why did the U.S. allow this technology to go to China for a supercomputer that can be used for defense applications?”

Oh, those poor wandering lambs. It kind of shoots down that old saying that, “There’s no such thing as a dumb question.”

In case you missed it, here is a quick recap of China’s chart-topping system, Milky Way-2.

Known as Tianhe-2, or Milky Way-2, which significantly belongs to China’s National University of Defense Technology, is configured with an unprecedented compute engine infrastructure consisting of 32,000 new Intel 12-core Xeon IvyBridge processors and another 48,000 Intel Xeon Phi coprocessors, totaling a staggering 3,120,000 compute cores and a peak performance of 54.9 petaflops.

Yes, it is powered by Intel microprocessors. The same off-the-shelf processor technology that’s available to anyone, including the United States. There is nothing in the processor technology that’s unique to China. What’s unique is how China was able to come so far – and so fast – with integrating an operating system, interconnect technology and other proprietary components into a framework that uses the same microprocessors that numerous other countries are using, yet do it so efficiently and with such impressive results.

“China doesn’t have to deal with the internal politics such as those keeping the U.S. from creating a unified exascale effort. Their government has determined the course and it’s an all hands effort to achieve their goal of exascale by the end of this decade. The only hope for the U.S. is a national program – a collaborative program – built on a holistic view that puts the DOE, the DOD, the NSF, and others under one umbrella of a national exascale initiative. In fact, hundreds of millions of dollars are being spent on a number of scattered HPC technology research agendas that are not tied together with a unified, long-term strategy. If we pull together other agendas, such as Big Data, and place them under one umbrella initiative to move research down parallel, intertwined paths, we could leapfrog right back into a commanding position and bring tremendous benefit to this nation.” *anon

“No one in Washington – the people entrusted with our national security and economic prosperity – should be surprised by China’s seizure of the No. 1 spot on the Top 500 List. This has been coming on publicly for years.” *anon

According to our esteemed HPC colleague, Professor Jack Dongarra, founding director of the Innovative Computing Laboratory at the University of Tennessee in Knoxville, “This Milky Way-2 is a machine on the path to exascale, but there are many challenges ahead and much research and a great investment is needed to get there. Perhaps this is a wake-up call to the US. Time will tell.”

We agree. This should (once again) be a wakeup call for the U.S. The implications – that some have called a ‘ threat’ to national security, economic competitiveness, and technology leadership – posed by China having the world’s most powerful computer, and the resulting high performance computing ecosystem that will trickle down from that accomplishment, is much bigger than just one system.

SO WHY ALL THE FUSS OVER JUST ONE MACHINE?
Think about the implications of accelerating your nation’s research agenda(s) with the world’s most powerful computer(s). The ability to design breakthroughs in transportation, both civilian and military, from automotive to rail to fighter jets and naval ships, or the ability to model, simulate, and ultimately build advanced research and manufacturing capabilities, resulting in pharmaceutical and medical breakthroughs that will drive multi-billion dollar industries. It’s not about just one machine. The so-called exascale machine is symbolic. The resulting eco-system will have dramatic economic impact. When exascale is achieved, all boats will rise.

When I’m Frustrated, I Often Repeat Myself
“China’s position of owning the most powerful computer on the planet should certainly not be a surprise to anyone in High Performance Computing, and should not be a surprise to any U.S. legislators. Have we already made this point? Dozens upon dozens of articles have been written over the past several years predicting this outcome as a reflection of a lack of U.S. vision, leadership and funding.“

In an article from The Exascale Report back in 2010, Gilad Shainer, Chairman of the HPC Advisory Council, provided this insightful prediction: “China for example is going their own direction — more or less — and targeting increased development within China. The HPC Advisory Council has a new paper, Toward Exascale Computing, that has just been published. It includes graphs showing where China was five years ago and where they are today. It’s a huge performance jump — a giant step. If this pace of progress is maintained, then the next number one system in the world is going to be in China.”

Within the next 2-3 years, it is very likely this impressive Chinese system architecture will be powered by Chinese-built processors and accelerators, and at that point, the U.S. will be in a difficult position.

While China is confident they will have an exascale system by the end of this decade, one HPC industry luminary, Horst Simon, Deputy Director of Lawrence Berkeley National Laboratory, is so confident the United States won’t have an exascale system by the end of the decade that he’s made a wager with a colleague, betting $2,000 that we’ll miss the goal.

A number of community leaders are conflicted about U.S. companies selling the necessary technology to Russia or China for the development of supercomputer systems.

“We don’t seem to have issues with selling to the EU or Japan. Does that mean we see China and Russia as threats to U.S. technology leadership – or threats in even more ways?“*anon

“When the Chinese finally build their exascale systems and no longer need the stepping stone processor technology from Intel that was used in their current top supercomputer, will that [Chinese] technology be only used in China or will it be marketed to the world? And if it is, and then ultimately adopted by numerous U.S. companies, can we rely on technology from another nation to power our society – from banking to health care to manufacturing to Wall Street – and still hope to maintain our freedom? We must have equivalent or even better technology to protect the economic interests of this country.” *anon

According to Dona Crawford, Associate Director for Computation at Lawrence Livermore National Laboratory (LLNL), “The amount of money we have to spend, and this is true for any nation, determines how aggressive any of us can be with our research and development efforts. We’ll make the best use of any funds we can procure. When you hear that one country is committing hundreds of millions of dollars, it’s actually very difficult to determine how much of that is actually going to specific research. But there is no question, China is serious about investing in this technology.

China wants to beat that 2020 deadline. And they are not just interested in a stunt machine, I think they have demonstrated in several different ways the importance of high performance computing to their country, to their national pride as well as to their national security and their economic prosperity. They have a very systematic, broad-based, sustained, approach to building out their HPC ecosystem. And they are making great progress.”

According to Rick Stevens, Associate Laboratory Director for Computing, Environment and Life Sciences at Argonne National Laboratory, testifying before the U.S. House of Representatives Science, Space, and Technology Committee’s Subcommittee on Energy, “China has announced plans to build more than a dozen supercomputing centers, with an announced goal of reaching Exascale capability by 2018.) Japan is planning a next-generation supercomputing project with an estimated budget of 110 trillion yen, and Europe has established PRACE (Partnership for Advanced Computing in Europe) to advance high performance computing and to re-establish an HPC industry in Europe.

Right now, our competitors are relying primarily on American technology to create these powerful machines. But increasingly, other nations are developing the expertise and technology to build supercomputers that could rival or even surpass American-made high performance computing systems.”

The Exascale Evangelism Tour
Luminaries like Rick Stevens, Horst Simon , Dona Crawford, Roscoe Giles, Dan Reed and others (just to name a few) have been actively engaged in exascale discussions, projects and research for more than seven years. They have been champions of the cause – and without them, and their perseverance we would be much further behind the curve than we are. But despite their efforts, and the dozens of other highly qualified spokespeople who have been part of what I refer to as the exascale evangelism tour, progress and commitment from the U.S. government has not been there. The latest victory goes to China.

MEANWHILE – BACK IN SILICON VALLEY
If there is one battalion of crusaders carrying the exascale standard, at least today, it’s Intel.

It’s no surprise the Chinese supercomputer uses the latest Xeon processor technology. The Top 500 list of the world’s fastest computers currently looks like an Intel customer list. 80 percent of the TOP 500 computers have Intel processors as the heart of their computation engines, and 98+% of the new systems on the Top 500 list use Intel processors.

The company has been demonstrating an unswerving focus on HPC for several years now, and the results are impressive. Hey, the Milky Way-2 is all the evidence we need of how important this technology is to HPC. Intel’s plans for its next generation 14-nanometer Knights Landing product will, according to the company, provide the ability for customers to use the new chip as a coprocessor or as a primary CPU.

As articulated in recent interviews with Intel’s Diane Bryant, Raj Hazra, and Alan Gara, Intel has targeted exascale and gives no indication of being swayed to change its course. The next generation Chinese supercomputers may replace Intel technology with something home grown, but it’s highly probable that the rest of the world will be making their own exascale plans based on an Intel roadmap.

Intel doesn’t like to include words like “war” or “fight” in their lexicon, but as we look at the global exascale battlefield from 10,000 feet, no one else seems better equipped or qualified to push the technological front line for the U.S.

But this battle takes more than a platoon or a regiment. It takes an entire army. Will DOE, OSTP, and OMB come together and provide the air cover and the reinforcements that are needed, partnering with industry to get the U.S. back on top?

CLOSING THOUGHTS
Perhaps the new leadership at DOE, under the command of Ernest Moniz will change all of this and get the U.S. marching in the right direction. To do so, he’ll need to win the confidence of his Captains, and he needs a strong Director for the DOE Office of Science – someone who understands the absolute, critical importance of extreme scale computing. But ultimately, the challenge Secretary Moniz and his troops face is a matter of budget priority. No one in the Senate or House has said they don’t believe in the importance of exascale-level computation, but the disagreement on how to get the necessary funds and where to trade off budget items requires someone who can bring several organizations with different priorities together, build a necessary army of crusaders, and then take the battle to Capitol Hill and secure the commitment of OSTP and OMB.

The entire global HPC community will be watching closely to see if Moniz has the strategy and the tenacity for rallying the troops and giving the U.S. a battlefield advantage.

Exascale Report Déjà vu

In The Exascale Report™ article from July, 2010 titled, “Will Exascale Drive an Unprecedented Level of Global Cooperation – Or Will It Fan the Fires of Global Competition?” we had an interesting quote from an anonymous source in China referred to as “Mr. Zheng.”

At that time, China’s leading HPC system was Nebulae at the number two slot on the Top 500 list of the world’s most powerful supercomputers. Nebulae was actually the fastest system worldwide with a theoretical peak performance of 2.98 petaFLOPS per second. That was three years ago.

Following is what Mr. Zheng had to say at the time:

“The highly competitive nature of the Chinese people, driven by a great sense of national pride. Nebulae is a proof point. It demonstrates China’s ability, and to a certain degree our desire to be recognized as a global technology leader. Fortunately, we have the resources to collaborate and compete aggressively in this race.”

The article closed with this quote from Zheng:
“We need exascale to drive scientific research. We need computational power at the exa level in order to better understand and protect this planet. And for me, personally, I think it would be great if the first exascale computer had a very large engraved tag that said, ‘Made in China’.”

We offer this closing illustration from an Exascale Report article dated January, 2012, If the U.S. doesn’t step up its game – the writing may already be on the wall.

Ernest Jeffrey Moniz (born December 22, 1944) is an American nuclear physicist and the current and the 13th United States Secretary of Energy, serving under President Barack Obama since May, 2013. He previously served in as the Associate Director for Science in the Office of Science and Technology Policy in the Executive Office of President Bill Clinton from 1995 to 1997 and in the United States Department of Energy, serving as Under Secretary of Energy, from 1997 to 2001.

Moniz is one of the founding members of The Cyprus Institute and the Cecil and Ida Green Professor of Physics and Engineering Systems, Director of the Energy Initiative, and Director of the Laboratory for Energy and the Environment at the Massachusetts Institute of Technology.

On March 4, 2013, Moniz was nominated by President Barack Obama to replace outgoing Energy Secretary Steven Chu for his second term. His appointment was confirmed by the Senate in a unanimous vote on May 16, 2013.

HPC and Exascale Evangelists mentioned in this article

Rick Stevens	Dona Crawford	Horst Simon
Dan Reed	Rosco Giles	Gilad Shainer
Barbara Helland	Paul Messina	William Harrod