Comments on: Video: Does the TOP500 Need to Evolve? http://insidehpc.com/2012/12/10/video-does-the-top500-need-to-evolve/ HPC News Without the Noise for Supercomputing Professionals | insideHPC Sun, 19 May 2013 10:46:54 +0000 hourly 1 http://wordpress.org/?v=3.1.1 By: Sid http://insidehpc.com/2012/12/10/video-does-the-top500-need-to-evolve/#comment-381689 Sid Tue, 11 Dec 2012 11:26:56 +0000 http://insidehpc.com/?p=33713#comment-381689 It can't really be said to be a controversy when everyone agrees it's a useless metric to measure capability by. It can’t really be said to be a controversy when everyone agrees it’s a useless metric to measure capability by.

]]> By: @hpcnotes http://insidehpc.com/2012/12/10/video-does-the-top500-need-to-evolve/#comment-381663 @hpcnotes Mon, 10 Dec 2012 16:53:01 +0000 http://insidehpc.com/?p=33713#comment-381663 There are three distinct points here: 1) "acquiring supercomputers based on potential TOP500 ranking hurts users" - I agree with this sentiment and am glad NCSA did this. 2) "the LINPACK benchmark used is not a valid performance benchmark of modern systems" - this is only partly true - LINPACK has ALWAYS been only ONE measure of a system - and in that way is still valid today - but the problem arises because some people treat it not as one of many metrics, but as THE metric. 3) Neither faulty use of the Top500 (e.g. procurement) nor the use of a single benchmark (LINPACK or any other single metric) mean that the Top500 is useless. The value - and correct use - of the Top500 is 500 biannual data points consistently measured over 20 years - not any one data point. There are three distinct points here:

1) “acquiring supercomputers based on potential TOP500 ranking hurts users” – I agree with this sentiment and am glad NCSA did this.

2) “the LINPACK benchmark used is not a valid performance benchmark of modern systems” – this is only partly true – LINPACK has ALWAYS been only ONE measure of a system – and in that way is still valid today – but the problem arises because some people treat it not as one of many metrics, but as THE metric.

3) Neither faulty use of the Top500 (e.g. procurement) nor the use of a single benchmark (LINPACK or any other single metric) mean that the Top500 is useless. The value – and correct use – of the Top500 is 500 biannual data points consistently measured over 20 years – not any one data point.

]]> By: RichB http://insidehpc.com/2012/12/10/video-does-the-top500-need-to-evolve/#comment-381660 RichB Mon, 10 Dec 2012 16:28:39 +0000 http://insidehpc.com/?p=33713#comment-381660 As described by Bill Kramer, NCSA did not submit a benchmark result for the Blue Water supercomputer to the TOP500 even though the system would have easily ranked in top ten. The reason cited by Kramer is that the LINPACK benchmark used is not a valid performance benchmark of modern systems and that acquiring supercomputers based on potential TOP500 ranking hurts users. Read more: http://www.ncsa.illinois.edu/News/Stories/TOP500problem/ As described by Bill Kramer, NCSA did not submit a benchmark result for the Blue Water supercomputer to the TOP500 even though the system would have easily ranked in top ten. The reason cited by Kramer is that the LINPACK benchmark used is not a valid performance benchmark of modern systems and that acquiring supercomputers based on potential TOP500 ranking hurts users.

Read more: http://www.ncsa.illinois.edu/News/Stories/TOP500problem/

]]> By: John Melonakos http://insidehpc.com/2012/12/10/video-does-the-top500-need-to-evolve/#comment-381659 John Melonakos Mon, 10 Dec 2012 16:25:31 +0000 http://insidehpc.com/?p=33713#comment-381659 Nevermind... I missed your previous post. Thanks for the info! Nevermind… I missed your previous post. Thanks for the info!

]]> By: John Melonakos http://insidehpc.com/2012/12/10/video-does-the-top500-need-to-evolve/#comment-381658 John Melonakos Mon, 10 Dec 2012 16:24:35 +0000 http://insidehpc.com/?p=33713#comment-381658 What is the controversy? What is the controversy?

]]>