Companies usually wind down the week before Christmas, so you don’t usually see them make a lot of strategic moves or announcements. And so it was with some marked astonishment that I received an anonymous tip that Oracle ceased development of Lustre right before the holidays. Not out of a job quite yet, Lustre engineers have reportedly been encouraged to apply for other positions within the company.
You might question the timing of this move, but what better way to bury a story than to pull the plug when everyone else is home for ten days singing Kumbaya?
No one at Oracle responded to requests for comment on this story, but that’s how the company works. When I went poking around the Lustre community for some kind of confirmation, not one individual was surprised about this development. And while most were reluctant to go on the record, they all had heard the same story.
Careful What You Ask For: Short Term Support of Lustre a Key Issue
As Oracle exits the Lustre business, support of current Lustre installations is a real short term issue that will need to be resolved quickly.
The National Labs may be able to hold their own, but they can not provide software support to other institutions. That will be the job of the many vendors that have or support Lustre on their storage for customers to buy. Open source organizations probably have no role to perform in providing support; they can perform a role in hosting open source code repositories, bug databases, and providing an open “forum” for customers and vendors to collaborate.
The good news is that, in 2010, market forces responded to the opportunity presented by the Lustre Limbo, with new companies forming (Whamcloud) and others like Xyratex staffing up to fill in the gaps.
Norman Morse, CEO of OpenSFS, had this to say:
As your article points out, there is a huge commitment to and dependence on the Lustre file system in the Supercomputing community. Because of the requirement for a vibrant Lustre system in the future and given speculation about possible changes in Lustre support, members of the community, including some who caused Lustre to be created, founded OpenSFS specifically to ensure Lustre continues to meet community requirements and remains the preeminent open source parallel file system for high performance computing. We are moving forward with this as our mission and we encourage all members of the Lustre community to join us – see www.OpenSFS.org.
Again as your article points out we don’t wish to comment or speculate on Oracle’s plans but OpenSFS have always stated our intention to fully cooperate with Oracle as they develop their business plan for Lustre. Through member dues, OpenSFS has resources at hand to make a major contribution to continued support and development of Lustre by funding Lustre support organizations.
The Lustre Silent Auction?
We can look forward to more Limbo for a while, but what happens next with stewardship of Lustre? Will Oracle quietly kill it like they did with OpenSolaris? Will they set the legacy code base free like they did with Grid Engine? Or will they just cash in and sell it?
Chances are that Lustre is being shopped around and we won’t hear a peep from the Dark Tower until a sale is announced. That could be months, and that kind of prolonged uncertainty would not be good for the Lustre community.
Now, if you’re wondering who might buy Lustre, you probably should look at who stands to make a lot of money supporting Lustre or selling disk storage systems that run it. I’m thinking DDN, Xyratex, or even Whamcloud. At last count, there was something like 100 Lustre engineers within Sun, so continued development is going to require deep pockets.
A Great Opportunity for the Lustre Community
As I wrote in a recent story about Lustre engineers joining Whamcloud, Oracle’s Lustre bumbling has rallied the Lustre community in a way that might not have been possible before. With the formation of HPCFS, OpenSFS, and the European Open Filesystems Group, the community has done remarkable job of organizing to ensure that the popular open source file system remains viable for their pending supercomputer plans.
I know what you’re thinking; since Lustre is open source, maybe it doesn’t need a corporate holding company. The National Labs helped spawn Lustre and they can just take it back.
Why There isn’t an Easy Answer
Herein lies the rub: according to the published Lustre roadmaps, the future of Lustre on Linux is the incorporation of ZFS. Who owns that code? Oracle. So this divorce will come with strings.
Speculation and second-guessing aside, Lustre remains important for lot of reasons. Half the systems on the TOP500 run it, and there’s no open source replacement out there at the moment that is ready for prime time. And while no one in the Lustre community would probably cry if Oracle fell off a cliff, any kid can tell you that Bumbles bounce.