A study by Hyperion Research finds that approximately 20 percent of HPC workloads are now running in the public cloud. There are many good reasons for this trend.
As enterprises move further down the path to digital transformation, they increasing rely on compute- and data-intensive applications driven by high performance computing systems. Historically, the vast majority of these applications ran in on-premises data centers, but today this is changing. Growing numbers of HPC applications are now being run in public clouds.
This was a key research finding highlighted in a new Hyperion Research white paper, “Bringing HPC Expertise to Cloud Computing.” In this paper, sponsored by Dell Technologies and AMD, Hyperion notes that prior to 2018, the percentage of HPC workloads running in the public cloud had plateaued at approximately 10 percent of the overall HPC workload portfolio. In 2019, based on Hyperion Research surveys and studies, the percentage of HPC workloads running in the public cloud jumped to about 20 percent.
That’s a notable shift, one that signifies the increasing maturity and acceptance of HPC solutions and services offered by cloud service providers (CSPs).
So what’s behind this trend? Hyperion says that a major driver of increased cloud-based HPC usage is the recent availability of hybrid and multicloud options offered by many CSPs. These offerings are designed to provide HPC users with a near seamless environment between on-premises hardware and counterpart cloud resources.
“As HPC applications grow in complexity, as in the case with the addition of new AI techniques to a wide range of application portfolios, the ability to easily burst into the cloud provides a more flexible, responsive computing environment,” Hyperion notes. “Likewise, multicloud capabilities are increasingly appealing to many HPC sites as they allow users to manage multiple third-party cloud resources from a single ‘pane of glass,’ a feature especially attractive to global corporations that use different CSPs scattered across different geographic locations.”
But that’s just part of the story. Here are some of the other drivers for HPC in the cloud, as called out in the Hyperion paper:
- Cloud offerings are becoming much more capable for HPC applications. CSPs and other companies are providing easier access models to clouds, and a larger set of application software is available in the cloud.
- Many CSPs now provide support specifically targeted for HPC workloads, including in-house HPC experts and services to help HPC users better understand the process of running their applications in the cloud.
- With the added HPC expertise from CSPs, HPC users have more resources at their disposal to effectively and properly use the right instances on the right platform for each of their jobs. Even better, the added capabilities of multicloud options make it possible for each application in a given portfolio to run in a separate environment tailored to its needs.
- Cloud platforms are becoming an increasingly important enabler in the emerging and high-growth markets of artificial intelligence and other high performance data analysis applications. Critical applications here include automated driving systems, precision medicine, affinity marketing, business intelligence, cyber security, smart cities and the Internet of Things.
Hyperion notes that HPC is crucial to AI developments. Moreover, the growth of HPC in the cloud, as well as the attention paid to HPC by CSPs, has resulted in many AI workflows being created and run in cloud environments.
“For some AI workflows, access to enough data for training can be challenging for on-premise execution or expensive to transfer from a cloud environment,” Hyperion says. “In these cases, data residing or collected in a cloud environment, from sensors or the combination of multiple different data sets, can be better run where the data exists.”
Fueling market growth
In a parallel trend, the accessibility of HPC in the cloud has helped fuel dramatic revenue growth in the HPC cloud market. With the jump in HPC workloads running in the public clouds, Hyperion updated its HPC in the cloud forecast to show an almost $1 billion increase in 2019 HPC cloud revenue.
“HPC in the cloud will continue to grow as barriers to entry, namely data security, data movement and cost, are addressed by CSPs, ultimately enabling end users to take full advantage of the cloud,” the firm says. “Today, the cloud is primarily a complementary solution to on-premise resources, especially with hybrid cloud offerings, but that could change in coming years as cloud capabilities increase.”
To learn more
For the full story, read the Hyperion Research white paper “Bringing HPC Expertise to Cloud Computing.” And for an even deeper dive, check out the Hyperion webinar “New Capabilities Drive Users to the Cloud for AI and HPC Workloads.”
