Data—the gold that today’s organizations spend significant resources to acquire—is ever-growing and underpins significant innovation in technologies for storing and accessing it. In this technology guide, insideHPC Special Research Report: Modernizing and Future-Proofing Your Storage Infrastructure, we’ll see how in this environment, different applications and workflows will always have data storage and access requirements, making it critical for planning to understand that a heterogeneous storage infrastructure is needed for a fully functioning organization. Most organizations require both High Performance Computing (HPC) technologies as well as other IT functionalities.
Introduction
Historically, when organizations need HPC capacity, it means a dedicated facility, dedicated system administrators, a separate budget and independence from the rest of the IT organization. The hardware, software and networking are probably put out to bid to a few select system integrators (or the systems vendors themselves), and a long and involved procurement process follows. This may involve specific benchmarks, architectural designs and compatibility matrices (what component worked with what other component). All these disparate parts and vendors and budgets result in one, less-than-ideal thing: a silo created specifically for the organization’s HPC infrastructure.
Mixing Workloads
Organizations, both commercial and research, now realize that the inclusion of HPC into the mainstream workflow has benefits that go beyond just finding an answer to a problem that requires an HPC environment. Data that is ingested for a simulation using HPC technologies can also be used for business intelligence applications. Large amounts of data that periodically are generated during a long-running computation can be used for more targeted real-time marketing. The ability to use the massive amounts of data that are input into an HPC environment or the result of simulations creates an opportunity for IT organizations to consolidate their infrastructures across functional groups, which can save money.
Components/Storage
HPC environments are designed to get the most out of the highest performing components available. The consideration of the choice of CPU vendor, number of cores, clock rate and overall performance for the given workload can be benchmarked and selected. Examples of CPU vendors for high-performing workload needs include Intel, AMD and ARM.
The performance and capacity of memory systems within an HPC computing environment are critical. The larger the high-speed memory, the more data can be held close to the CPU, increasing overall performance. Hundreds of gigabytes to terabytes of memory are required to hold the enormous amounts of data that an HPC application will access, or to hold the data that a real-time business intelligence application will need. Acceleration for applications has made a huge difference in the performance for certain use cases. A thousand-fold increase in the overall performance has been noted and the range of accelerators that are tuned to specific tasks is increasing. Heterogeneous HPC is becoming a reality, with specific processors and architectures for specific domains and application requirements. This includes the choice of storage systems as well.
However, in many cases the choice for the storage component is an afterthought, with the idea that all storage is the same. But the ability to retrieve and store data with considerations for performance, capacity, scalability and security should actually be a priority . After all, without access to the data for any reason or for any amount of time, overall business operations will be compromised.
Over the next few weeks we will explore these topics surrounding storage infrastructure:
- Introduction, Mixing Workloads, Components/Storage
- Challenges with HPC Today and How it is Changing, Industry Trends
- Industry Innovators, Storage Considerations
- Value, Seagate as a Trusted Partner, Next Steps
Download the complete insideHPC Special Research Report: Modernizing and Future-Proofing Your Storage Infrastructure, courtesy of Seagate.
