John Gustafson from A*Star will keynote the third annual StartupHPC Workshop on Nov 14 in Salt Lake City. At StartupHPC, Gustafson will present on Technology Disruption. Known to many in the HPC community as the father of Gustafson’s Law, John Gustafson was kind enough to sit down with us to share his perspectives on the current state of high performance computing.
insideHPC: First we’d like to hear about your new position at the A*STAR center in Singapore. What are you working on there?
John Gustafson: A*STAR took great interest in my work on unum arithmetic, and offered me a visiting scientist position to develop those ideas further. That effort has already resulted in the “type 2” unum idea that should be very hardware-friendly and far faster than floating-point arithmetic, and the idea of customized, application-driven numerical systems.
My Singapore stint also involves being a Professor at National University of Singapore, the top university in Asia and a remarkable institution. I do a little teaching (Parallel Processing, Advanced Computer Architecture), but NUS mainly provides me with grad students to help me with the unum research.
It’s probably the best job I’ve ever had in my life. And I really love Singapore!
insideHPC: Your talk is entitled “Technology Disruptions and Opportunities.”
Can you give us a preview your talk?
John Gustafson: I think I’ve figured out the pattern of winners and losers in the computer startup business, based on 34 years of working at private companies of every size from 2 to 100,000 employees. There are classic signs when a big company is about to stumble and probably fail completely, usually because a small company that they ignored or mocked turns out to be eating their lunch. The established companies may give lip service to Moore’s law, but whenever their product marketing (prices and feature set) tries to hold steady the way non-computer industries do, that’s a sure sign that they’re doomed. Which means opportunities for startup companies. My talk will be a collection of examples of this, drawn from personal experience.
insideHPC: The presentation outline mentions a “fundamental change in computer arithmetic”
that is happening right now. What is that change and why is it a “painful” change to
John Gustafson: Case in point! One of the signs of an entrepreneurial opportunity is when a new idea is greeted with, “Yes, that’s a better way to do things, but we’ve got FAR too much invested in doing things the old way. So no one will ever change.” That was the reaction to parallel computing for HPC, you may recall. In the 1980s, people thought there was too much investment in vector processing, a la Cray mainframes and FPS array processors, and it was hopeless to think that HPC users would rewrite their codes to use a cluster of processors with distributed memory. I see the same thing happening now with respect to 1980s-style floating-point arithmetic. Because transistors are now about a million times cheaper than they were when the IEEE 754 committee defined floating-point arithmetic guidelines, computer arithmetic is overdue for a makeover that will be just as wrenching as the change to parallel computing. It’s very similar to the change we are going through right now in moving from the 8-bit ASCII character set to Unicode.
I’m finding a groundswell of interest in unum arithmetic as an extension to floats, or even a complete replacement for floats, so there are plenty of “early adopters” on which to base an HPC startup. It could be the equivalent of a Thinking Machines or an nCUBE for our era. One startup, REX Computing, has already indicated the intention to incorporate unum arithmetic into the next generation version of its ultra-efficient processor design.
insideHPC: Do you think the HPC market is place where Startups can prosper and grow?
John Gustafson: Absolutely! But there is a trap, a fallacy that is specific to HPC startups that I wish we would learn to avoid. It goes something like this: First, build a very simple processor with phenomenal speed at matrix-matrix multiplication and almost nothing else (with far too little memory bandwidth to support the arithmetic units in general), touting its potential LINPACK score to impress potential customers and investors. Next, provide the bare minimum software needed for it to run that benchmark and make some sales. Try to stay in business while working on applying the very special-purpose architecture to general HPC problems. That stage is usually accompanied by the company complaining about their compiler, which would run EVERYTHING fast if only it were sophisticated enough to exploit the brilliant architecture thought up by the hardware designers. Then the company usually dies or is acquired for pennies on the dollar. I’ve been watching HPC startups falling into that trap for over three decades, and it keeps happening. I’ll mention SiCortex by name since it no longer exists, but I’ll not name any other names because some of those companies are still trying to make it.
I don’t talk about that trap in my presentation at StartupHPC, but instead concentrate on approaches that have shown a pattern of success.
insideHPC: In your experience, what is the best way to spot a Startup opportunity?
John Gustafson: Think about companies that seemed laughable when you first heard about them, but that seemed to be selling their product nevertheless. I will show my age with my examples, but the Apple II computer was like that. So was the very first car that Honda introduced, having only been a motorcycle maker up to that point. More recently, some silly little company came up with the “Acorn RISC Machine” processor for personal computers, a while back; well, you now know that as ARM, and there’s nothing silly about it. The best startup opportunities are when you can be the first to disrupt a market the way Clayton Christensen describes in his classic book, “The Innovator’s Dilemma”: from below, with something that almost looks like a toy version of products offered by the mainstream manufacturers. Now that I think about it, Floating Point Systems was an example of this, since it was viewed as “The Poor Man’s Cray,” but its rise to success was quite steep with that approach.
The best way to spot a Startup opportunity is to notice an established vendor that has gotten too comfortable with its large market share, and thinks it can raise profit margins and resist the downward price pressure of Moore’s law. A startup can become the disruptor-from-below of that fat-and-happy company.
John Gustafson is the father of Gustafson’s Law, which gives the theoretical speedup in latency of the execution of a task at fixed execution time that can be expected of a system whose resources are improved. He is also author of The End of Error: Unum Computing, which explains a new approach to computer arithmetic: the universal number (unum). The unum encompasses all IEEE floating-point formats as well as fixed-point and exact integer arithmetic. This new number type obtains more accurate answers than floating-point arithmetic yet uses fewer bits in many cases, saving memory, bandwidth, energy, and power.”