Sign up for our newsletter and get the latest HPC news and analysis.

Hundreds of thousands of threads? Yes, with Erlang

Multicore.info with the pointer to a blog post by Bartosz Milewski on some of the problems with threaded programming in the mainstream languages

Thread model based on heavy-duty OS threads and mutexes has its limitations. You can ask server writers, or Google for “thread per connection” to convince yourself. Servers use thread pools exactly because of that.

Thread pools are an admission of defeat for the thread model….[but]…The more fundamental problem with threads has its root in memory sharing. It seems like sharing offers great advantage in terms of performance, but sharing requires locking. It’s a well known fact that locking doesn’t scale (or compose). Between races and deadlocks, it’s also extremely hard to get right.

A solution? Erlang!

Erlang interpreter provides lightweight processes (so lightweight that there’s a benchmark running 20 million Erlang processes).

And some refreshing honesty

Why don’t we all switch to Erlang?…Functional paradigm is as alien to our everyday experience as quantum mechanics….[and] …Messages have to be copied. You can’t deep-copy a large data structure without some performance degradation, and not all copying can be optimized away.

This post marks the first reference to Erlang in over 2,000 posts at this blog. I smell a t-shirt!

Resource Links: