Does the U.S. need a national exascale initiative? Are DARPA’s extreme scale program and DOE’s exascale program complementary or competitive? And what about Moore’s Law? Is exascale needed because Moore’s law is coming to an end? Want to get answers to the really hard questions? Then you need to go to the top. That’s just what we did. In this exclusive interview coordinated with insideHPC, The Exascale Report interviews Intel CTO Justin Rattner.
The Exascale Report: Some people are saying that exascale is needed because Moore’s law is coming to an end. Do you agree with this? What is the future of Moore’s law?
Justin Rattner: Such a belief stems from a fundamental misunderstanding of Moore’s Law. Gordon Moore, Intel’s cofounder, only predicted that the number of transistors on a chip would double roughly every two years. Unfortunately, many people think Moore’s Law forecast that chip performance will double every two years, which it does not. I’ve occasionally heard the performance-based version referred to as Joy’s Law after Bill Joy of UC Berkeley and, later, at Sun Microsystems. While Joy’s Law may have held true for some period of time, it stopped being true almost a decade ago.
Until then, performance was scaling with transistor count, but chip power was increasing at a faster rate. Eventually, switching those many millions of transistors at multi-GHz speeds drove power levels beyond what was economical and practical to cool. Another contributor was leakage power as a result of scaling the sub-100 nanometer gate lengths. The two factors in combination caused a leveling off of clock frequency, and a dramatic slowdown in single core performance.
The architecture focus quickly shifted to multiple cores as a way to generate higher performance without running as hot. Cutting clock frequency just 10 percent saved almost 50% in power. You could literally get almost twice the aggregate performance from two cores as you got from one single core running only 10% faster. It was this observation that ushered in the era of multi-core microprocessors and drove interest in many-core processors such as GPUs.
Extreme scale research is asking a different question: can we find a way to use all of those transistors we will have when we reach the 8nm node without blowing the per chip power budget. It’s an enormous challenge, but we would not even be thinking about such extreme designs were it not for Moore’s Law continuing to drive ever higher transistor budgets.
Let me make one more comment about Moore’s Law: it doesn’t say anything about what technology will be used to keep growing the transistor budget. We did reach the end of usable silicon-gate transistor scaling at 65 nanometers. Leakage was simply too high and performance was not improving at the historical rates. Intel invented a new kind of transistor, called Hi-K metal gate, to solve the scaling problem for at least the next three generations of process nodes, but we did it on the same two year cadence Moore’s Law has been on for decades. Nobody noticed that Moore’s Law had ended for silicon gate, but began for Hi-K metal, because we made the transition without missing a beat. The same has not been true for others in the industry who are struggling to bring Hi-K metal gate technology to market.
TER: DARPA has an extreme scale program; DOE has an exascale program. Are these competitive or complementary?
Rattner: These programs are, in fact, complementary. DARPA is looking beyond current computer architecture and taking a risk with the hope of high reward. DOE is looking at the scientific and engineering computational tasks and ensuring that the US is equipped with sufficient computational performance to meet the national needs. A good analogy would be that of scouts in an army. DARPA is sending out scouts into unexplored territory, some scouts will no doubt fail return, but others will come back with very valuable information. DOE, in contrast, is like the army. With the information provided by the scouts, they will be able to move forward with higher confidence and a greater likelihood of success. We intend to use our DARPA partnership to scout out new circuit topologies and new computer architectures, and we intend to work with DOE to deliver the systems based on those learnings that keep the U.S. at the forefront of high performance computing in latter part of this decade.
TER: Why has Intel decided to engage in government funded programs after a long absence?
Rattner: We have continued to work with the U.S. government on various programs in the national interest. Information security and silicon-based optical communications are just two examples. I would also note that we were part of several of the HPCS proposals way back when, but there was a strong bias against commercial-off-the-shelf technology, and none of those proposals were funded. We continued to deliver technology to the HPC marketplace, even though we received no direct government support, and Intel-based machines have come to dominate the Top500 rankings.
Intel also believes we’re at an important inflection point in computing systems. Most of the low hanging fruit in terms of energy efficiency has been picked. The next order of magnitude improvement is going to require much more fundamental changes and is likely to bring a new set of challenges. Such situations create an ideal environment for innovation.
With exascale we have an opportunity to fundamentally re-think circuit and logic design, program execution models and the architectures that support them, and the programming methods for such innovative machines. Such advances are difficult to achieve in the existing horizontally segmented computer industry because it requires the coordinated action of many companies with differing interests working together to achieve the complete hardware/software co-design. Exascale is the kind of engineering challenge that can bring these independent actors together to solve a really big problem.
TER: Intel is investing what appears to be significantly in exascale research in Europe. Why Europe? And how does your European exascale research balance out with what you are doing a) in other countries, and b) in the U.S.?
Rattner: Europe is doing some smart things both at the EU level and the individual country level. They are willing to co-fund joint exascale research labs with industry. I’d love to see DoE do the same thing in the U.S., but they continue to keep the manufacturers at arm’s length. More importantly, they don’t see exascale as a research problem. They see it as a procurement problem. As long as the department holds that view, the exascale efforts outside the U.S. will likely progress at a much higher rate.
TER: Who at Intel really calls the shots when it comes to the company’s exascale research efforts and overall exascale or extreme scale strategy and direction? Though various sources and from different meetings, we’ve heard five different people describe their roles as being “in charge of Intel’s exascale strategy.”
Rattner: I don’t know who all your sources are, but the Intel situation is really very simple. Exascale research is owned by Intel Labs and that includes the DARPA-funded UHPC research program. Exascale development is owned by the Datacenter Group under Kirk Skaugen. Raj Hazra runs the HPC team under Kirk, replacing Richard Dracott.
As in any large company, the two groups are not always on the same page, but that’s actually what we want. The researchers must have the freedom to explore many interesting technical possibilities to fail fast, as we say, while the developers need to focus on what can be built on a high-confidence schedule and be competitive in the market. We argue and debate a lot, but in the end we both want to deliver the best technology to our customers.
TER: What is your perspective on China — and their supercomputing efforts? In a recent random survey I personally conducted, asking people where they thought the first exascale system would appear, the responses looked like this:
- China
- Europe (probably Germany)
- U.S.
- Russia
Rattner: Well, let’s not get too carried away with all this talk about first to exascale. Most of us are old enough to remember how the Japanese were going to dominate HPC, but they chose the wrong architectural approach and paid dearly for their mistake. And, when you say first to exascale, do you mean first with the technology or first to write the check to buy the technology. There’s a big difference.
The U.S. is almost certain to lead in the technology for exascale, but it remains to be seen whether the U.S. will have the political will to write the first check. Your list looks more like a prediction on who will win that competition. That said, I need to remind everyone that exascale is an unbelievably difficult problem. While there are several promising paths to get there, none is guaranteed. We have a long, long way to go. I wouldn’t spend too much time right now debating the first check topic.
TER: Does the U.S. need a National exascale initiative? What are the consequences to Intel, to science and to the USA — if we do not have a Manhattan or Apollo style initiative to lead in Exascale and computing in general? Does it matter, since companies like Intel, IBM and HP for example are now essentially global?
Rattner: HPC is different sort of challenge than landing a man on the moon or building an atomic bomb. The latter missions have well-defined endpoints. HPC does not. It’s the never ending story which makes it hard to get the public excited about HPC because, as we say at Intel, fast enough never is. With HPC you’re back every few years asking for money to build the next big machine. It gets old very fast. The real trouble comes when the technical challenge of building the next machine is overwhelming, as it is with exascale, and business as usual doesn’t work anymore.
There are two ways out of this predicament. One is to rethink HPC as something more akin to development of the next generation of military aircraft. The government understands that there is no commercial market for military aircraft and funds all the R&D to get them designed and built. They even fund multiple companies to design the same kind of aircraft and then hold a fly-off at some point late in the development. DoE should give serious thought to this approach with exascale. There is almost no market for so-called “capability machines” which makes them a perfect candidate for the military aircraft funding model.
The alternative is to take HPC to the high volume marketplace, and siphon technology off the best of those products to build the next big HPC system. My SC09 keynote was precisely aimed at this approach. I noted that it’s clear to anyone who cares to open his or her eyes that HPC does not have a viable business model. To put it another way, the emperor has no clothes. Market growth is slow, stock prices languish, and VCs are reluctant to fund new entrants. Perhaps a bigger concern is the fact that young people are writing apps for Android and iOS4, not petascale machines.
The choice is therefore quite clear: you either treat HPC like weapons systems and let the government shoulder all the costs or you find a high volume market for HPC. Unfortunately, neither DoD nor DoE has shown any willingness to do either. HPC is still treated as any other “data processing equipment” acquisition. For a fraction of the cost to the taxpayer of paying the entire R&D expense associated with HPC, a much more modest investment in creating a healthy HPC industry would seem to be a very reasonable alternative. If we could only create some compelling applications of cloud computing that would drive ups its FLOPS demand by several orders of magnitude, we wouldn’t’ need the government to pay for exascale.
My suggestion for a high-volume HPC application was a simulation-based Web. Intel continues its efforts toward that goal through efforts such as OpenSim. It would be nice if DoE would direct some of their money at the problem, but I’ve literally had no expression of interest from them since my keynote. I even felt that some at DoE were annoyed by what I had to say. Again, the emperor has no clothes.
TER: “Reaching exascale in this decade will require a level of international collaboration beyond anything we’ve ever seen.” Is this really possible? When we start to get closer — isn’t it far more likely that collaboration will turn to competition as countries start to compete for the bragging rights associated with fielding the first exascale systems.
Rattner: Again, are you asking about who will write the check or who will have the technology? The technology will certainly be a global undertaking. Even within Intel we have labs in the U.S., Europe, and Asia all working on some aspect of HPC, and each of those labs has engagement with HPC efforts in their home countries.
TER: During the quest for TeraFLOPS, we used to speak more often of the Grand Challenges; what would you say Exascale enables that would make this monumental effort worthwhile — from Intel’s and your perspective?
Rattner: I presume we’ve used the term grand challenges so many times that it now tends to fall on deaf ears. I don’t see exascale as defining new problems as much as it reasserts the endless need for greater computing power. HPC has become a bit like shampoo. You know, rinse and repeat.
While the focus of the HPC community in targeting exascale is aimed at enabling scientific discoveries in fields such as materials, astrophysics and microbiology, the required breakthroughs in computing technology have broad application across the entire computing continuum. It is that fact that drives Intel’s interest. Let me give you two examples.
First, consider the exascale supercomputers now being expected for late this decade by the HPC community. If we simply scaled one of today’s petaFLOPS supercomputers, which is capable of 1015 floating point operations per second, to exascale (1018 FLOPS) levels, we’d need a battery of nuclear power stations to supply its six gigawatts (6GW) of electrical power. With a useful limit of about 20 megawatts (20MW) of power in an HPC datacenter, we need roughly a 300x improvement in total system energy efficiency to build a practical and deployable exascale supercomputer.
Second, consider an end-of-the-decade smartphone with extreme sensing capabilities requiring 100 gigaFLOPS of computing power. Without the anticipated breakthroughs in extreme-scale technology, such a phone would need a very, very big battery delivering 600 watts of instantaneous power. Even if we could somehow deliver that much power, it would be very unpleasant to hold the phone, not to mention the battery, in your hand. If we are successful in developing the extreme-scale technology required for exascale computing, 100GF would consume a mere two watts of power or even less. That’s one wicked-smart smartphone with a market measured in the hundreds of millions of chips.
If the extreme scale technologies will allow us to deliver compelling solutions at both ends of the spectrum, our shareholders will be very, very happy.
For related stories, visit The Exascale Report Archives.
