Is open source the way to go for HPC software?

Print Friendly, PDF & Email

The enterprise software space, particularly at the applications level, is almost entirely ruled by commercial software. There are some open source alternatives, but for most part, large corporations are usually willing to pay top dollar to get support for mission-critical applications. Add to that the fact that ERP and CRM packages are among the biggest software systems ever produced, and it becomes apparent why enterprise software tends to not be open source.

As for the technical crowd, many of the truly innovative uniprocessor packages tend to be commercial as well. These are the programs that are interactive and feature graphical interfaces, etc. For most technical users, these will be sufficient.

Then there are the customers who require large-scale number crunching, and thus rely on parallel computing. The fact is, there are very few customers who require this type of platform at the moment. Because the market is so small, there are so few vendors in this space, which in turn leads to inadequate commercial products for those who do need them.

In this space, it is easy to see where open source initiatives are crucial. Most parallel computing customers tend to be universities or government labs (though a few financial groups such as derivatives traders and hedge fund quants may require this level of computing sophistication). The universities can work together to develop software, which they do quite well, for example, with Open MPI.

Open source initiatives further make sense in this area namely because of the possibility for user-contributed material. (Indeed, Web 2.0 pundits blog about this phenomenon since dot-coms essentially act as a forum for collaboration.) One of the reasons Linux has worked so well as an open source product is that many of its users know how to program. The same appears to be true of many parallel computing users.

It could be the case that there is much greater demand for parallel software than currently believed among vendors. That demand, however, will probably remain latent until something is done to greatly ease the use of the product. Until that becomes wide-spread, most parallel computing software will by necessity remain open source.