Does TOP500 Even Matter? Bill Kramer Rationalizes the Blue Waters No-Show

Print Friendly, PDF & Email

As covered here on insideHPC, one of the biggest surprises of the new TOP500 list was not about who topped the list, but the fact that NCSA decided that getting on with the business of real supercomputing was more important than beating their chest. In a candid blog post this week, NCSA’s Bill Kramer describes exactly why his team decided not to submit a LINPACK result for Blue Waters, a recently installed massive Cray supercomputer that would certainly rank among the fastest machines in the world in anyone’s book.

Why does the HPC community let itself be led by a misleading metric? Some computer centers, driven by the relentless pressure for a high list ranking, skew the configurations of the computers they acquire to maximize Linpack, to the detriment of the real work to be performed on the system. The TOP500 list and its associated Linpack benchmark have multiple, serious problems. This column is too short to deal with all of them in detail. But a few of the issues and possible solutions are briefly listed below.

The Blue Waters supercomputer at NCSA from Cray may be the most-anticipated supercomputer in the history of the industry, so much the fact its omission from the TOP500 calls into question the very validity of the list itself. Is Kramer rationalizing a decision to not release a bad LINPACK result? Probably. Is his criticism of the LINPACK benchmark a view that many in the industry agree with? Absolutely.

In my view, the rest of us in the HPC community have to decide: Do we want to acknowledge that the Emperor has no clothes?

Read the Full Story.

Trackbacks

  1. […] NCSA decided not to list their new Blue Waters supercomputer on the Top500 list, and this has been much discussed in the press. While Titan for now holds the Top500 crown, the Blue Waters Lustre file system is rumored to be […]

Comments

  1. It’s about time someone with some actual influence make that decision. Well done, that man. It’s not that the Top 500 is bad, it’s still a measurement, but it’s not that relevant. It’s become a marketing tool that hasn’t evolved with the industry.

  2. NCSA’s excuse is trite. According to my source @ NCSA, the real reason seems because Blue Waters is not uniform: there are CPU nodes and CPU-GPU nodes, and there is no Linpack code which can use both. If BW is just a smaller version of Titan, I am pretty sure NCSA will be more than happy to submit the result to Top500.

    Although there has been incessant attacks on Linpack as the yardstick of supercomputers, the truth is Linpack is the most stringent stress test on many aspects of a supercomputer; not only does it consume the most energy when it’s running, it has high requirement on floating-point performance, memory bandwidth, and interconnect latency. It is also able to run on many different architectures and problem sizes. You really cannot find another synthetic benchmark which has all of these.