Building concurrency in Yahoo! apps with Erlang

From time to time I’ve posted about Erlang on this site. It’s the Good & Plenty of programming languages. Everyone has heard of it, but not many folks like it, and it’s always a little interesting when you see someone at the movie theater order the box they’ve had under the counter since the first run of Episode IV: A New Hope.

Anywho, Dr. Dobb’s ran an article this week on a use of Erlang at Yahoo! (which has experimented with many functional languages for subsystems that support their Internet applications) to rebuild one of their core data-gathering applications

Harvester is a data acquisition component at Yahoo! that acquires data from various providers concurrently through a pull model from drop boxes via various protocols. Feeds can either be scheduled or fetched on demand.

While Harvester was originally written in Perl, Erlang’s high-level concurrency constructs — along with OTP design principles — make it an ideal platform for building reliable, fault tolerant, and scalable applications like Harvester. The resulting service is more scalable, available, reliable, and able to comply with tighter service-level agreements (SLA) on a lighter code base and less expensive development efforts. In this article, we compare how the two systems perform side-by-side on a single box and showcase the higher ROI provided by the Erlang stack.

I’m interested: any Erlang out there in classical HPC applications?