Exascale Research Gains Attention in Russia; Formalized Global Collaboration Efforts Emerging

Print Friendly, PDF & Email

From the earliest discussions of exascale development, many have posited that success will only be achieved with extensive global cooperation, bringing the most advanced thinking to bear on the many challenges presented by extreme complexity. Identifying those global resources is a continuing goal of The Exascale Report. We have covered various US and EU initiatives and projects as well as the efforts of the IESP to create a cohesive exascale roadmap.

In this article, we gain a Russian perspective on exascale development in an interview with Vsevolod Opanasenko, CEO and Chairman of the Board of T-Platforms. Opanasenko has guided the company from its founding in 2002 to its current multinational status, which includes T-Platforms, T-Services, T-Massive Computing and T-design divisions and offices in Hanover, Germany, Moscow, Russia, Campbell, California, and Taipei, Taiwan. He was recently honored as one of HPCWire’s People to Watch for 2011.

T-Platforms Group is working in close cooperation with leading academic institutions, particularly Moscow State University (MSU) to develop exascale technology in Russia. The company has delivered two systems that are the largest known in Russia, Chebyshev and Lomonosov. The latter is number seventeen on the Top500 as of this writing, achieving over 300 TeraFLOPS with 35,360 cores.

Bob Feldman: As the leading developer of HPC systems in Russia, How would you describe the status of HPC systems in Russia today, their configuration and uses?

Vsevolod Opanasenko: There is still a very wide gap between high-end and average HPC installations in Russia. On the one hand, T-Platforms typically develops at least one top-performance system a year, which has grown from 625 nodes in 2008 to 5,878 nodes in 2011. The rest of the market lags considerably behind in terms of installation size, although certain positive trends can be observed. If in 2008 a medium system node count just exceeded 20, in 2010 it grew to approximately 50.

In terms of usage, most supercomputers in Russia are deployed at academic and research sites. Industrial installations are still scarce. However, in 2010 we entered into a number of contracts with industrial companies, and this year new systems will emerge in this segment.

Feldman: How does the role of an HPC vendor in Russia differ from the role in the EU or the US?

Opanasenko: T-Platforms typically delivers its systems complete with external storage and on-site power and cooling infrastructure, which is rarely the case in other markets. Deep expertise in HPC data center construction and applications is something which a customer requires in emerging marketplaces like Russia, and which gives us an edge over other system vendors, enabling customers to avoid dealing with multiple suppliers. More often than not, we also deliver third party application packages to address the customer’s specific needs, over and above necessary system software.

Feldman: Last year, T-Platforms expanded into Western Europe and is now launching a sales effort in the US. What are your goals in these new markets and how does this relate to exascale development?

Vsevolod Opanasenko

Vsevolod Opanasenko

Opanasenko: T-Platforms is a member of PROSPECT and European Open File System alliances. We are also working as one of the experts for the European Exascale Software Initiative project.

The German office is largely devoted to developing robust bonds with the local Tier-1 supercomputer sites in order to conduct joint research and development into future European HPC technologies. Rather than trying to compete with European integrators in low-end and mid range segments, we have applied a long-term strategy in Europe, which implies a deep technological partnership with customers such as Forschungszentrum Juelich (FZJ), Leibnitz Rechenzentrum (LRZ), Federal High Performance Computing Center Stuttgart (HLRS), CSC (Finland) and others.

One of our first EU deliveries last year was a TB2-TL GPU-based system for Leibnitz Rechenzentrum, which will be used to explore novel water-cooling technology aimed at improving the energy efficiency of an HPC center. Another important technological partnership was initiated with Heidelberg University for joint development and manufacture of a low latency EXTOLL interconnect.

T-Design in Taipei is focused on system design and manufacturing, and the U.S. office is concentrating on sales and market development, striving to build strong partner relations in order to meet the demand for our latest high-end GPU-based offering. We could estimate that within three to five years, the number of our systems installed abroad will probably exceed those supplied to [the Russian] market. We are hoping to see our first US installations this year.

Feldman: You have a system in the Top 20 now. When will T-Platforms have PetaFLOPS systems installed and how will they be used?

Opanasenko: Our first PetaFLOPS system is an 800 TFlops expansion of the Lomonosov supercomputer deployed at the Moscow State University. The system has now reached 510 peak TFLOPS and is based on T-Platforms TB2-XN blade systems equipped with Intel Nehalem and Westmere processors and QDR InfiniBand interconnect. We are now expanding it with TB2-TL NVidia-based blades. With 17 TFLOPS of double precision peak performance in a 7U chassis, it takes only 8 racks to add 800 TFLOPS of extra power to Lomonosov, bringing the total hybrid system peak performance to 1.3 PetaFLOPS.

Lomonosov is devoted to developing large models scaling up to several hundred TFLOPS. To name just a few computational problems being solved for industry, Lomonosov performs CFD simulations for aerospace and atomic power engineering customers, seismic modeling, and future supercomputer component design simulations.

Feldman: What formalized exascale initiatives are in place in Russia today? What is T-Platforms involvement in those Russian exascale initiatives?

Opanasenko: Supercomputing research has gained a lot of attention in Russia recently. The National Supercomputer Technology Platform concept is targeted at exascale. As part of that program, T-Platforms has signed a MOU with Moscow State University to co-develop a future exascale system. We believe that within a year Russia will have a formalized exascale objective, where we will most actively collaborate. Exascale barriers, in particular in terms of application software, will be tackled across a number of other sites including Federal agencies like Rosatom, and major Russian universities. We also believe that deeper integration between Russian and worldwide exascale efforts is of crucial importance.

Feldman: IESP has put forth road maps for exascale development and EU initiatives such as PRACE actively fund projects to develop the exascale software stack and tools. What is T-Platforms involvement in EU or other international initiatives such as IESP and how do you see that role evolving?

Opanasenko: T-Platforms strives to be among the first vendors embracing the entire scope of exascale technology, and we are devoting a great deal of attention to research into this area, seeking collaboration both in Russia and Europe. Moreover, we are convinced that the best results will be gained if Russia and Europe unite their efforts in overcoming technological barriers to exascale. We are cooperating with PRACE through PROSPECT and EOFS, and expect to become more deeply involved in next gen prototype development for its members this year.

We also form part of EESI (European Exascale Software Initiative) community, where supercomputer sites and vendors work together to develop the European exascale roadmap. We are very open to cooperation and flexibility in adjusting our R&D to meet the specific needs of our European HPC customers. We are actively looking at opportunities to invest in emerging technologies and create shared IP. We also intend to work towards making Russian supercomputer R&D space and infrastructure part of a broader European environment.

Feldman: What percentage of your research and development is related to exascale today? How do you see that changing over the next few years?

Opanasenko: It is fair to say that all of our high-end product line development is about making steps towards exascale. This is currently more than half of our entire R&D effort. We also have several research projects under way that are specifically targeted at technologies with 3 to 5 year perspectives. We estimate that within 3 years, the majority if not all of our efforts will be aimed at this goal.

Feldman: The computational expertise in the Russian technical community is well known. What particular strengths and skills does the T-Platforms team, and the research community you work with in Russia, bring to the difficult task of achieving exascale performance?

Opanasenko: Our core expertise lies in power and cooling technologies, networking, dense packaging, scalable OS and system management solutions. In Russia we also have very strong teams devoted to developing compilers and tools. The Russian HPC research community has been traditionally strong at creating efficient algorithms and mathematical methods, which are being implemented in a number of highly scalable original codes, especially in such areas as CFD.
Our top-level priorities at T-Platforms include research into low power multicore processor technology, fiber optics both at inter-node level and silicon photonics, reliability, interconnect and communication support in software, exascale API environments, OS and management software.

Feldman: How would you summarize your view of exascale development? Where are we now and can we get to exascale in 10 years? Is it worth the huge expense and effort that will be required or would the world be better off if we focus on better programming models, tools, multicore optimization, and other problems, rather than bigger scale?

Opanasenko: Practice shows that major advances in hardware architecture propel software development, and vice versa. It seems hardly feasible to try to rewrite legacy codes or develop new, highly scalable ones unless there is hardware that promises truly extraordinary performance increases. Although it now looks like we are at the very start of exascale development, we are convinced that we will arrive at this level of performance somewhere between 2018 and 2020, at least in terms of hardware – as sure as we eventually made our way from TFLOPS to PFLOPS.


Overall, this interview paints a picture of a Russian HPC market with a growing high end, a pronounced “missing middle” and a preference for single source vendors with soup to nuts capability from data center infrastructure to provision of 3rd party applications.

Opanasenko points out the talent and strengths of Russian computer science and an eagerness to integrate with EU initiatives, supporting the notion that Russia may play a significant role in exascale development. Russian strengths may emerge to contribute innovative solutions for power, cooling, networking, dense packaging, scalable OSs and system management solutions. As the leading vendor, T-Platforms and Vsevolod Opanasenko himself are likely to figure prominently in whatever role Russia plays in global cooperative development of Exascale.

About the Author

Bob Feldman

Bob Feldman is Senior Contributing Editor of The Exascale Report and President of HPC Marketing, a marketing and sales development consultancy specializing in HPC and other advanced technologies.

Contact Bob at bob@hpcmarketing.com or www.hpcmarketing.com.