There has been a lot of interest in the fate of the Lustre, so I caught up with Bill Boas to talk about the popular open source file system and the newly-formed HPCFS International Foundation for Lustre.
insideHPC: Maybe we should start at the beginning, Bill. How did you get started with Lustre?
Bill Boas: You may not know this Rich, but I was the original Lustre program manager for Lawrence Livermore (LLNL) and on the Tri-lab Path Forward team that managed the development of Lustre for the first three years. I was the Contract Technical Representative, so I’m also very familiar with the development and testing resources needed for Lustre on Linux as well as the business model that has been employed so far for the evolutionary aspects of Lustre. I had almost daily interaction over several years with CFS before it became part of Sun.
After leaving LLNL, we, System Fabric Works, did development engineering work on Lustre for CFS and Sun. We’ve also performed deployment and support for Lustre at several customer sites. We also made Lustre work in a wide area for a defense department project that we can’t talk too much about. Other than the TeraGrid, this project is the only other wide area deployment of Lustre that we know about. The project requires a widely dispersed federated file system for multiple clusters and storage on which image analysis and real time actionable data can be shared. We, System Fabric Works, were part of the team that made Lustre work over 6000 miles between and across continents. We also were on the team that deployed Lustre on RedSky at Sandia – No. 10 on the Top 500. So I’m definitely not a newcomer to Lustre.
insideHPC: So what prompted the formation of HPCFS?
Bill Boas: When we learned what Oracle’s plans were for Lustre on Linux at the last Lustre User Group meeting, a number of people realized that the vast majority of the Lustre customers on Linux needed to be reassured that there was a path to the future for them. The success of the Open Fabrics Alliance has shown us that people really appreciate an effective open source community where there is a focus on collaborative development, testing and distribution of code that meets the requirements of the common interests of the customers.
In order to achieve that, one needs to have an egalitarian organization so that decisions are made by consensus. Every member participates and feels that they have a voice in how the organization works. In my mind, the situation with Lustre on Linux in the early summer this year, called for the formation of a traditional kind of open source organization.
So I founded HPCFS as a non-profit foundation in July. And immediately vendors and customers started showing strong interest because they understood that the foundation provides the forum for vendors to understand many customer’s requirements, for vendors and customers to collaborate on joint development, testing and releasing as well as mail lists and databases for problem reporting, correcting and patch distribution.
We’ve tried to keep the costs reasonable for the several hundred Lustre customers. Corporate members can join the board for as little as $10,000 with an initiation fee of $3000. Academic and Government can participate as members for $5000, annually and there are individual memberships for as little as $200.
The purpose of collaborating this way is to add to the resources that Oracle has working on Lustre on Linux and to form a community that contributes not only original code, but also patches when they find a problem.
It will also bring additional resources to the community for testing. My early experience with Lustre was that testing was the most important aspect with Lustre because it’s essentially a real-time system on a cluster. That’s what a network wide, scale-out file system turns out to be, and timing of events is absolutely crucial. Events don’t take place on a consistent, predictable timeline in a real-time system and they don’t do that in Linux, either.
Persistent testing at some level of scale is the only way to find the bugs. So the community needs to bring to the future of Lustre a wide range of environments where such testing can take place.
insideHPC: What you’re talking about sounds great, but, after seeing what happened to the OpenSolaris project, do you think Oracle will pay attention?
Bill Boas: Yes. Oracle has said publicly that they wish to participate and work with HPCFS. They are welcoming HPCFS to use all of their internal tools and their mailing lists. Plus they’re asking intending members of HPCFS to bring additional technical resources to help them with Lustre on Linux. Oracle is also participating on our planning calls — we find their participation encouraging.
The first community building component we’re working on is to ensure that we participate with Oracle to make sure that together the community responds to customer needs by making additional resources and testing available so we can collaborate and work with Oracle very closely, including some of the companies involved with Lustre such as Cray, Dell, Fujitsu, HP, SGI, TeraScala, WhamCloud, Xiotech, Xyratex, etc.
insideHPC: So what is the current status of the HPCFS organization?
Bill Boas: We’re up and running. We have a web site at HPCFS.org. The bylaws, members agreements, and contribution agreements that we think will be necessary are there in draft form. Everything is based what on other open source communities have done, and have proven workable for successful collaboration.
There’s one other major aspect; HPCFS is a non-profit corporation incorporated in Delaware. It’s actually called the HPCFS Software Foundation, and we are getting ready to apply to the IRS for 501(c) 3 exemption, just like the Apache, Mozilla, and Linux Foundations. And what that will enable is that an entity or individual that supports the foundation’s work can donate funds and benefit from a tax deduction.
insideHPC: I get it. A non-profit is a lot different than a business association where dues and donations would have to be considered a business expense. That’s a real advantage. That brings me to my next question: Would you welcome OpenSFS as a member of HPCFS?
Bill Boas: Yes we would. As far as HPCFS is concerned, everyone is welcome to join the foundation. I see OpenSFS as having an opportunity to access funds to drive some of the high-end, extreme scale features of Lustre, and that will be very beneficial for the customers in the top10 or so of the Top500. HPCFS is a much broader and more inclusive type of organization that should include those users as well as the wider community.
insideHPC: Is there a European effort going on as well?
Bill Boas: Yes. In Munich on September 27 there was a meeting where a group of vendors and users held their first meeting. They are forming a cooperative that can apply for European Union funding. I was in Europe a couple of weeks prior and that’s what they want to do. We’ve given them some suggestions about governance and that sort of thing, and we’d like to see them form a non-profit, cooperative corporation in Europe that has the same kind of ability to take tax-deductible contributions. We’re looking at their group as a close affiliation.
I think the overarching message I wish to convey is that the Lustre community is vibrant and strong. Some people are concerned about the future of Lustre on Linux as the Sun products and services prior to the acquisition are rationalized by Oracle to match its ongoing business interests. HPCFS aims to collaborate with all the key stakeholders worldwide to ensure that the scale, performance, reliability and serviceability of Lustre on Linux remains production-worthy near term and for the foreseeable future.
insideHPC: Well Bill, this really sounds to me like the right move for the Lustre community. Now that you’ve described the mission here, what would you like the people who are interested to do next?
Bill Boas: The call to action is: Learn about more about HPCFS bylaws and agreements. If you like what we’re doing and it meets your interests, then help ensure Lustre’s sustainability by becoming members, contributing technical knowledge, and making testing resources available
Pleased visit the web site (under development) where the time and location of the HPCFS meeting, probably on Monday, at SC10 will be posted. Also, come by Booth #4755 in the Exhibits to give us feedback, become a member, or learn the latest news.
insideHPC: That sounds great, Bill. I applaud what you’re doing and good luck with the new organization.