Russian RSC Group deploys ‘hot water’ cooled supercomputer at JINR

Today the RSC Group from Russia announced the deployment of the world first 100% ‘hot water’ liquid cooled supercomputer at Joint Institute for Nuclear Research (JINR) in Dubna. This new system was named after Nikolay Nikolayevich Govorun, former Director of JINR’s Computing and Automation Lab in 1988- 1989, who worked on development of information technologies in this international scientific organization since 1966.

It’s great to note that we launch the new heterogeneous supercomputer named after professor Govorun at JINR’s Information Technology Laboratory in the year of 60th anniversary of commissioning of the first Ural-1 supercomputer at our institute in 1958. Our scientists and research groups now have a powerful and modern tool that will greatly accelerate theoretical and experimental research of nuclear physics and condensed matter physics,” – said Vladimir Vasilyevich Korenkov, Director of the Information Technology Laboratory of the Joint Institute for Nuclear Research.

The new supercomputer of the Joint Institute for Nuclear Research has been developed and deployed for the Theoretical Physics Laboratory (TPL) named after N.N. Bogolyubov and the Information Technology Laboratory (ITL) with support of JINR management. The project was completed by RSC Group with Intel specialists support.

The project involves plans of radical acceleration of comprehensive theoretical and experimental research in the fields of particle physics, nuclear physics and condensed matter physics. This includes deployment of NICA accelerator complex based at JINR to recreate quark- gluon plasma – a special state of matter of the Universe in first moments after the Big Bang. NICA mega-project started in 2013 and it is planned to commission this accelerator facility in 2020.

New supercomputer of the Joint Institute for Nuclear Research will enable resource- intensive massive parallel calculations to solve problems of lattice quantum chromodynamics for studying the properties of hadronic matter with high energy density and baryon charge and in presence of strong electromagnetic fields. It will greatly accelerate simulation of collision dynamics of relativist heavy ions and provide new resources to study the properties of highly correlated systems in the physics of new materials. It will also be used to develop and adapt software for NICA mega-project for new computing architectures, create an innovative hardware and software environment for high performance computing and simulating complex scientific problems and to train IT specialists in all relevant areas.

New JINR’s supercomputer is a natural development of HybriLIT heterogeneous platform with significant increase of performance. Theoretical peak performance of the new supercomputer is about 500 Teraflops (in double precision).

The new supercomputer includes high dense and energy efficient RSC Tornado solution based on Intel server products with 100% ‘hot water’ direct liquid cooling. The Joint Institute for Nuclear Research deployed new universal RSC Tornado computing cabinets with record-breaking power density and precision liquid cooling balanced for continuous usage of high temperature cooling agent (up to +63°C at cabinet entry point). JINR’s specifications required optimal cabinet temperature mode with +45°C constant cooling agent temperature at node entries (+57°C peak temperature).

‘Hot water’ mode enabled deployment of free cooling solution (24x7x365) using only dry coolers running at up to +50°C ambient temperature and complete elimination of freon circuit and chillers. As the result, average power efficiency factor (PUE) is less than 1.027. The system uses less than 3% of consumed power for cooling, which is an outstanding result for HPC industry.

Computing nodes of JINR system are based on Intel server products: the most powerful 72- core Intel Xeon PhiTM 7290 processors, Intel Xeon Scalable family processors (Intel Xeon Gold 6154), Intel Server Board S7200AP and Intel Server Board S2600BP, Intel SSD DC S3520 with SATA interface in M.2 form-factor and the newest high-speed Intel SSD DC P45xx drives NVMe interface and 1TB capacity.

High-speed data transfer between JINR’s supercomputer nodes uses innovative Intel Omni-Path Architecture communication technology providing up to 100 Gbps non-blocking switching rate. It is based on 48-port Intel Omni-Path Edge Switch 100 Series with 100% liquid cooling to provide high efficiency of the cooling system in ‘hot water’ mode and the lowest TCO of the system. Intel Omni-Path Architecture meets current requirements of resource-intensive user applications and provides sufficient throughput reserve for the future.

Innovative monitoring and management system based on RSC BasIS software stack guarantees high availability, redundancy and simplicity of usage of computing systems based on RSC solutions. It can be used to manage single nodes and the entire solution, including infrastructure components. All elements of the system (computing nodes, power supplies, hydraulic regulation modules, etc.) have an integrated management module providing broad capabilities for detailed telemetry and flexible management. Cabinet design supports replacement of computing nodes, power supplies and hydraulic regulation modules (with redundancy) in hot-swap mode without interruption of system operation. Most components of the system (such as computing nodes, power supplies, network and infrastructure components, etc.) are software-defined, and this significantly simplifies and speeds up initial deployment, maintenance and future upgrades of the system. 100% liquid cooling of all components ensures their longevity.

We are glad that RSC’s innovative HPC solutions will help to advance the Russian science and to improve the efficiency of international scientific research cooperation that has a long history in JINR,” – said Alexander Moskovsky, CEO of RSC Group.

Sign up for our insideHPC Newsletter