You know WETA Digital’s work: they did the special effects for the Lord of the Rings series, among other stuff. As you might imagine, all that rendering takes computers.
HP announced during Dresden that the company has installed 100 TFLOPS of rendering capablity in 4 different clusters that rank 219-222 on the current Top500 list. If they were ganged together, WETA would rank somewhere in the first 30 of the Top500. While I don’t know for sure why WETA configured the system in this way, a large customer of HP tells me that the company often encourages customers to break systems up in this way when the aggregate system won’t pierce the very top of the list in order to boost the proportion of the Top500 slots they hold. Interesting.
WETA is using the new ProLiant BL2x220c blade that lets users cram 12 TFLOPS in a rack about which I wrote in detail here. This blade is part of HP’s launch of its new Scalable Computing & Infrastructure group. As I wrote in that article, that group came from two groups within HP
…the HPC team with which we are all familiar, and HP’s Scalable Datacenter Infrastructure team. The SDI team has spent the last two years out of the public eye at HP, focused exclusively on a small group of large media and internet services companies, like Weta Digital (the team that did special effects for the Lord of the Rings series), Fox Interactive Media, and snapfish.com (HP’s photo sharing site).
And there’s the WETA connection. The HP systems replaced WETA’s previous IBM BladeCenter systems that held ranks 157, 481, 493, and 496 on the June 2006 list. HP’s site has a few articles referencing their WETA Digital work here and here.
The Top500 shows that the config of each ranked system is a HP Cluster Platform 3000 BL2x220 with 624 Intel EM64T Xeon L54xx (Harpertown) 2500 MHz (10 GFlops) Quad Core processors. I hear from people familiar with WETA’s installation that HP installed a total of 10 racks for these clusters, or 2,560 sockets. Since the Top500 configuration shows 2,496 sockets in each cluster, this leaves 16 sockets doing front end and I/O activities. A familiar configuration.
Given the amount of work that rendering takes and demand for WETA’s skills in Hollywood, its not hard to imagine an upgrade to the capability that they’ve already installed. Growing the configuration will be challenging, however, given the power and cooling needs of systems at this scale. There is a lower wattage processor in this configuration that could save the company on infrastructure costs.
According to this report at Forbes in June, New Zealand is in a prolonged drought, and with 60% of the country’s power coming from hydroelectric plants, power use is a big concern over there. If you’re from the area, can you confirm this?
I’ll watch the next couple lists for you and see how WETA evolves its infrastructure. It should be interesting.