WETA quietly building an HPC powerhouse

Print Friendly, PDF & Email

You know WETA Digital’s work: they did the special effects for the Lord of the Rings series, among other stuff. As you might imagine, all that rendering takes computers.

HP announced during Dresden that the company has installed 100 TFLOPS of rendering capablity in 4 different clusters that rank 219-222 on the current Top500 list. If they were ganged together, WETA would rank somewhere in the first 30 of the Top500. While I don’t know for sure why WETA configured the system in this way, a large customer of HP tells me that the company often encourages customers to break systems up in this way when the aggregate system won’t pierce the very top of the list in order to boost the proportion of the Top500 slots they hold. Interesting.

WETA is using the new ProLiant BL2x220c blade that lets users cram 12 TFLOPS in a rack about which I wrote in detail here. This blade is part of HP’s launch of its new Scalable Computing & Infrastructure group. As I wrote in that article, that group came from two groups within HP

…the HPC team with which we are all familiar, and HP’s Scalable Datacenter Infrastructure team. The SDI team has spent the last two years out of the public eye at HP, focused exclusively on a small group of large media and internet services companies, like Weta Digital (the team that did special effects for the Lord of the Rings series), Fox Interactive Media, and snapfish.com (HP’s photo sharing site).

And there’s the WETA connection. The HP systems replaced WETA’s previous IBM BladeCenter systems that held ranks 157, 481, 493, and 496 on the June 2006 list. HP’s site has a few articles referencing their WETA Digital work here and here.

The Top500 shows that the config of each ranked system is a HP Cluster Platform 3000 BL2x220 with 624 Intel EM64T Xeon L54xx (Harpertown) 2500 MHz (10 GFlops) Quad Core processors. I hear from people familiar with WETA’s installation that HP installed a total of 10 racks for these clusters, or 2,560 sockets. Since the Top500 configuration shows 2,496 sockets in each cluster, this leaves 16 sockets doing front end and I/O activities. A familiar configuration.

Given the amount of work that rendering takes and demand for WETA’s skills in Hollywood, its not hard to imagine an upgrade to the capability that they’ve already installed. Growing the configuration will be challenging, however, given the power and cooling needs of systems at this scale. There is a lower wattage processor in this configuration that could save the company on infrastructure costs.

According to this report at Forbes in June, New Zealand is in a prolonged drought, and with 60% of the country’s power coming from hydroelectric plants, power use is a big concern over there. If you’re from the area, can you confirm this?

I’ll watch the next couple lists for you and see how WETA evolves its infrastructure. It should be interesting.

Trackbacks

  1. […] behind kid-friendly features like The Incredibles, Toy Story, Ratatouille, and Monsters Inc. As recent purchases by film companies like Weta Digital indicate, film making is increasing an HPC […]

  2. […] Noir left a comment about a fire on that story (comment here). I won’t reproduce the comment in its entirety here since it includes the entire […]

  3. […] to get out in front of controversy in New Zealand where the energy envelope is pretty tight as we reported last year. Either way, nasty bit of truth stretching there by IBM.AKPC_IDS += "7006,"; Posted in HPC by […]

Comments

  1. Not many people are talking about the Fire that happened out at WETA. http://computerworld.co.nz/news.nsf/news/D378E7E589536D7DCC2574CC000AAD18

    Something “sensitive” happened at Weta Digital’s datacentre in Wellington about two weeks ago.
    Computerworld had been told an HP blade server caught fire, bringing out the fire brigade.

    Partly true, according to a PR spokeswoman at Weta. She says a “sensitivity” caused a hot spot in an old computer room, about to be decommissioned, which triggered an automatic call-out by the fire brigade. Asked to define “sensitivity”, she couldn’t. “But there is no story,” she said.

    Computerworld sources say the call-out was around 3am, caused by the failure of the power supply in the chassis of a high-density blade. “They lost all 32 servers in the chassis,” a source says, “but because they have so many servers it didn’t shut down the business.” The equipment was replaced under warranty.

    Weta Digital commissioned its new extreme density datacentre this month.