Thierry Pellegrino is vice president and general manager of Dell EMC High Performance Computing.
In this special guest feature, Thierry Pellegrino from Dell EMC writes that data analytics powered by HPC & AI solutions are delivering new insights for research and the enterprise.
For years, companies have been collecting data, but now, with the help of artificial intelligence (AI) and powerful analytics capabilities, they have the opportunity to get more out of it. However, with this opportunity, AI and analytics also have become big-data challenges that are changing how organizations and industries handle their data.
Many organizations are turning to high-performance computing as the solution, resulting in a wave of new ways to leverage HPC, new skill set requirements and new approaches for an era defined by volumes of data.
The Relationship Between AI and HPC
Analytics and AI, with their range of uses, require immensely powerful processes across compute, networking and storage, and organizations benefit most when compute is closest to the origin of data—wherever it resides. AI workloads, such as machine learning and deep learning, are being built atop HPC infrastructure to best support this demanding compute and data-intensive nature. As a result, more companies are increasingly using HPC solutions for AI-enabled innovation and productivity. For example, HPC technologies are being used to enable high-performance data analytics for training machine learning models, enabling researchers and organizations alike to gain insights and understanding from the vast influx of digital data.
ZEFF, Inc.’s AI database is a great example of what AI and HPC can achieve together. ZEFF pioneered the development of a groundbreaking AI database for unstructured image, audio and video data. It needs a highly reliable, enterprise-class infrastructure to train algorithms for artificial intelligence. With HPC, ZEFF has been able to solve tens of millions of image problems for people in a day or less instead of what previously took weeks or months to accomplish.
Organizations, such as Mastercard, also are leveraging AI on HPC systems in new ways—processing large data sets at lightning speed to identify and prevent fraudulent transactions.
Innovation in HPC technology and how it is applied continues to break new ground. Today, HPC workloads are becoming more data centric, adding AI technologies, advancing the capabilities of traditional HPC modeling and simulation. In the next few years, HPC technologies, such as HPC-enabled machine learning training, will go from experimentation models to production models. These technologies can be deployed for inferencing, honing-in on and automating items with the greatest ROI.
Preparing the Market and Workforces for HPC
This new confluence of data analytics and AI in research and enterprise, enabled by HPC solutions, creates entirely new possibilities and demands of IT. Providers need to be ready for the wave of people, who want to take advantage of technology to gain new insights, create new lines of business, and automate for speed and efficiency. Training and support for collecting and curating data, developing and training AI models, and deploying trained models will be key.
Users also need to develop the comprehensive set of skills needed to make this all work together in harmony. Applications and infrastructure must quickly and easily scale as data scales, meaning jobs are going to change as data grows and AI algorithms and tools evolve rapidly. We need to create a community that enables HPC to grow with people that understand its value. The good news is that data science is hot, and HPC experts are seen as critical members of the IT community, but we’ll need to have businesses collaborate more closely with universities to ensure the future IT workforce is prepare for these ongoing changes as well as the additional trends that will push the boundaries of AI and HPC.
Pushing the Boundaries of HPC
The technologies and trends that will push the boundaries of HPC potential are right around the corner,
and with them will come greater demand for HPC technologies. For example, the Internet of Things (IoT) is ushering in the new era of connected data at volume. With the advent of 5G, there will be infinite streaming possibilities from IoT devices–from watches to autonomous cars–that require HPC technologies to process vast amounts of data. This will be a goldmine for AI and the organizations leveraging those systems.
Also, in the past few years, we have seen smart living become increasingly popular, but with IoT and AI, smart living will hit new highs. We will see smart buildings, schools, factories, hospitals and venues to become the norm. Institutions and industries of all types will need to tap into HPC technologies to handle the evolving landscape, which is becoming overwhelmingly inundated with analysis and automation.
This increasing demand for HPC technologies, new trends in AI, and new use cases for both, will trigger an abundance of insights to be gained. HPC is clearly no longer reserved for large companies or research organizations. It is meant for those who want to achieve more innovation, discoveries, and the elusive competitive edge. With HPC available to organizations for every field and for every workload that needs performance, we’ll be able to accelerate innovation and achieve new levels of understanding through AI and analytics.
Thierry Pellegrino is vice president and general manager of Dell EMC High Performance Computing.
