In this podcast, the Radio Free HPC Team asks, “What are we going to do with 40+ AI chips?”
This week, the team looks at AI chips again, this time motivated by an article in EE Times about once such chip, Graphcore, and touts it as “the most complex processor ever” with some 20 billion transistors. The VC-backed company out of the UK is also valued on paper at $1.7b, gaining it the coveted “unicorn” status, apparently the “only western semi-conductor unicorn.”
This being one of 40+ such AI chips (and that may be conservative), the odds of success are tough and the task formidable. But even if only 2 or 3 of such chips become successful, that’s already a significant disruption to the market.
The Graphcore chip is 16nm, 1.6GHz, and comes in a PCIe card at 300W. You can stack 8 of these in a 4U chassis, so 2.4 kW just for those.
Highlights include:
- After a mini-rant about respected publications succumbing to clickbait, the team talks about how cooling will be an issue and calls again for more clarity in performance metrics since the chip is rated at 125 TFlops but we don’t know at what precision. Shahin reminds the team of his suggestion to clarify things by including precision in the metric, like DFlops for double precision, and then S for single, H for half, and Q for quarter precision.
- Henry talks about how hard it is to build and test complex software like this despite Shahin’s view that the modern software stack is too high so the chip need only be concerned with a couple of layers, codes are new and open to getting recompiled, it’s increasingly open source, cloud providers and large customers have the wherewithal to do the job, and traditional HPC customers have the willingness to do the work if performance enhancements are there.
