Where do old supercomputers go after the new machines are installed? At Durham University in the UK, they move on to a whole new mission in computational discovery for space science.
COSMA6 Supercomputer
Researchers specializing in astrophysics and cosmology, particle physics and nuclear physics at Durham University and from across the UK can now take advantage of an extended HPC service. The DiRAC Data Centric HPC system installed at Durham University has been enhanced by the deployment of COSMA6, a machine with 8,000 Intel Sandy Bridge cores and 4.3 petabytes of storage ‘upcycled’ from another system previously located at the Hartree Centre in Daresbury. This additional resource was needed to maintain the international competitiveness of the research community served by DiRAC for the next 12 months. In recognition of this, DiRAC was able to secure funding from the Science and Technology Facilities Council (STFC) to make use of the generous gift from Hartree and transfer and re-install the system at Durham University.
COSMA6 enables researchers to extend large-scale structure simulations of the evolution of the universe, analyze data from gravitational wave detectors, and simulate the Sun and planets in the solar system. COSMA6 is live and operational from April 2017.
The University’s COSMA6 service, combined with its existing COSMA5 service, now contributes over 14,000 cores to the national computing facility, DiRAC 2.5. DiRAC is a facility for theoretical modeling and HPC-based research in particle physics, nuclear physics, astronomy and cosmology, areas in which the UK is world-leading. Since its launch in 2010 DiRAC has received £29 million in funding from the Government.
The upcycled cores of COSMA6 added to the DiRAC Data Centric system installed at Durham University began life as a HPC machine at The Hartree Centre, Daresbury in 2012, but, as a consequence of upgrades, the Centre no longer had space to house the machine in its data centre. The staff of the Institute for Computational Cosmology (ICC) at Durham University worked with HPC, storage and data analytics integrator OCF, and the specialist server relocation and data centre migration specialist Technimove to dismantle, transport, and rebuild the HPC machine at its new home.
We had a very similar setup to The Hartree Centre; we had an IBM machine as did they; both of our older systems arrived in the UK within a day of each other in April 2012; we have a new machine room in Durham so we had the space, cooling and power and we also needed the cores to help boost our contribution to the national DiRAC system,” explains Lydia Heck, Technical Director at the Institute for Computational Cosmology (ICC) based at Durham University.
The rebuilding of the machine and its integration into the current DiRAC systems was an excellent example of collaboration between the customer and the IT partners. As half of the storage runs on DDN’s own Lustre parallel filesystem, Exascaler, and the other half on Intel’s Enteprise Lustre, the storage elements had to be installed in three racks that were re-cabled by OCF (and supported by DDN), and three re-cabled, and supported by the University. This complex integration of the different storage elements has been successful and the users see a seamless storage service.
Heck continues: “While it was quite an effort to bring it to its current state, as it is the same architecture and the same network layout as our previous system, we expect this to run very well.
The new HPC system at Durham University is a testament to the skills of all involved in the project who were able to re-install a second hand cluster, add to it new RAM memory and design a solution that will prove invaluable to the research community”, comments Julian Fielden, Managing Director of OCF. “We have a long history of working with Durham University so we’re really pleased to have been involved in such a unique project.”
