An Open Letter to the HPC Community

SPONSORED POST

We’re releasing PBS Professional under an open source license… why?

Bill Nitzberg, Altair

Bill Nitzberg, CTO, Altair

More than 20 years ago, I was just starting my career in HPC at NASA, where, along with a great team, we developed the original PBS software. Today, PBS Professional is among the most-used technologies for HPC job scheduling. But scheduling is hard — every site has unique processes, unique goals, and unique business requirements. In fact, there are at least a dozen different software packages that are in use across the globe for HPC job scheduling, in traditional HPC centers and in both private and public clouds. A dozen? Why?

Looking back over the last couple of decades, I believe a big reason is that the HPC world is really two worlds:

  • The public sector – researchers and scientists from universities and national labs are breaking new ground and pushing the limits of HPC – natural collaborators, early adopters and risk takers, which leads to a strong preference for open source software, and
  • The private sector – engineers and business people from corporations are building products and delivering services leveraging HPC as a tool for competitive advantage – natural competitors, later adopters and risk-averse, which leads to a strong preference, often a requirement, for commercial software.

Because the community is split, innovations do not easily flow from one sector to the other, and rather than advancing the state of the art, efforts are wasted re-implementing the same old capabilities again and again. This is a HUGE missed opportunity.

Altair is making a big investment toward uniting the whole HPC community to accelerate the state of the art (and the state of actual production operations) for HPC scheduling. We are dual-licensing PBS Professional®, both open source and commercial. We are opening the full core of the software, not just a weak subset or older version. We are continuing to offer a hardened, commercial version for commercial customers. We are committing to being aggressively open and community oriented. We are adding staff and reorganizing our engineering organization to behave as one of (hopefully) many contributors to the project. We are using community-accepted best practices (e.g., Github, JIRA, open discussions for new features). We are joining the OpenHPC project with PBS Pro. And, we are focused on longevity – creating a viable, sustainable community to focus on job scheduling software that can truly bridge the gap in the HPC world.

Excited? Honestly, this is one of the most exciting activities I’ve been involved in since bringing PBS Pro out of NASA in the first place. Getting the software in shape for release has required a lot of effort. We are on track for releasing open source PBS Pro 14.0 in mid-2016. It’s not only opening the source code, but also creating the tools and processes for nurturing a community – documentation, ticketing system, continuous integration tools, wikis, email lists, etc. We want to “do it right”. Of course, once we go live, I’m sure we’ll learn a great deal (from people like you), and we will work hard to best support the needs of the community.

Obviously, we cannot do this alone!

We want your input, your help; we want you to be part of the PBS Pro community. Please sign up to learn more at www.pbsworks.com/opensource.

To learn about the future of the open source community, request your copy of the IDC Technology Spotlight paper at www.pbsworks.com/IDCpaper.

Bill Nitzberg is the CTO of PBS Works at Altair. With over 25 years in the computer industry, spanning commercial software development to high-performance computing research, Dr. Nitzberg is an internationally recognized expert in parallel and distributed computing. He has served on the board of the Open Grid Forum, co-architected NASA’s Information Power Grid, edited the MPI-2 I/O standard, and has published numerous papers on distributed shared memory, parallel I/O, PC clustering, job scheduling, and cloud computing. In his spare time, Bill tries to reduce his pack weight for his long-distance hiking trips.