Sign up for our newsletter and get the latest big data news and analysis.

Intel to developers: you're not doing it right [UPDATED]

Peter Bright comments over at ars technica on a recent Intel blog post observing that developer efforts to adapt applications to 4 or 8 cores are not enough.

No, the future holds tens, hundreds, or thousands of cores, and developers are going to have to bite the bullet and write programs that will scale to such systems.

This recommendation came in a recent blog post, aptly titled “Unwelcome Advice.” Having to write programs of that scale on systems with hundreds of processors is difficult and totally alien to the vast majority of developers out there. The blog post describes two ways that developers are trying to accommodate increasingly parallel processors. Some are trying to move piecemeal, targeting first two or four cores, then perhaps extending it to eight, maybe even 16. Others are jumping headlong into writing software that can run on innumerable processors.

This is a true nightmare for developers, most of whom do not have the first bit of parallel programming experience or training. The process will be expensive and error prone, resulting in customers having to be convinced to pay more to get less in the short run. The only hope here is that someone will come up with the tools to institutionalize the expertise the HPC community has been hoarding inside its small group of programmers with dubious hygiene habits for the past 6 decades.

[UPDATE: Michael has a good blog post on this as well, and draws in a bit more of the context than the ars technica piece does.]

Resource Links: