In this video, NCSA researchers employ innovative IO technology from LSI in the Dark Energy Survey, which seeks to answer one of the most fundamental questions in astrophysics today; Why is the expansion of the universe speeding up?
NCSA provides data management for Dark Energy Survey that will collect digital data from astronomical observations in a database expected to grow by 400GB daily over the one and a half year life of the project to approximately 200TB of raw image data.
The prototype storage system that we developed for the National Center for Supercomputing Applications (NCSA) is designed around the latest LSI MegaRAID CacheCade technology, which allows solid-state drives (SSDs) to be used as RAID controller cache. This helps overcome the I/O bottleneck plaguing storage technology today so that DES can transfer the approximately 400GB of data that will be generated each day by the new telescope in Chile to storage systems in the United States. The plan is to eventually add more than one controller per node so that the controllers can also pool their SSD-based cache. Initial tests showed significant performance gains.