Astronomers Using iRODS to Wrangle Massive Survey Data

Print Friendly, PDF & Email

renciAstronomers are using iRODs data technology to study the evolution of galaxies and the nature of dark matter.

By partnering with data specialists at UNC-Chapel Hill’s Renaissance Computing Institute (RENCI) who develop the integrated Rule-Oriented Data System (iRODS) researchers and students now have online databases for two large astronomical data sets: the REsolved Spectroscopy of a Local VolumE (RESOLVE) survey and the Environmental COntext (ECO) catalog.

The RESOLVE Survey combines new optical and radio spectroscopy with archival multi-wavelength photometry to perform an unprecedented census of gas, dark matter, and stars in galaxies and their larger groupings, spanning nearly five orders of magnitude in spatial scale. The ECO catalog is designed to complement RESOLVE by providing a similar, but purely archival census within an order of magnitude larger volume.

“This is about going from rags to riches,” said Sheila Kannappan, PhD, associate professor of physics and astronomy at UNC-Chapel Hill and principal investigator for the RESOLVE project. “We went from having raw data with no ability to query it to databases that are searchable, expandable and flexible with version control.”

Both databases use iRODS, the integrated Rule-Oriented Data System, to store and retrieve galaxy information, flexible image transport system (FITS) files, and corresponding metadata. iRODS is free open source data management software used by thousands of businesses, research centers, and government agencies worldwide for flexible, policy-based data management that provides long-term preservation and federation. iRODS is developed and maintained by a software engineering team at RENCI with support from the iRODS Consortium, a membership-based foundation organized to sustain iRODS development. iRODS developer Justin James worked closely with Kannappan and her research team to incorporate the two astronomical surveys into the iRODS platform.

Data doesn’t mean much if it can’t be accessed and used, and we are proud that iRODS played a part in making these important data sets available,” said iRODS Consortium Executive Director Dan Bedard. “It’s great to play a role in helping scientists make discoveries and educating future scientists.”

The new RESOLVE and ECO databases premiered this week at the 2016 American Astronomical Society meeting in Kissimmee, FL.

Sign up for our insideHPC Newsletter