Sponsored Post
This article continues our series exploring the HPC transition to the cloud, and what your business needs to know about this evolution. Compiled in a complete Guide, we cover cloud computing for HPC, considerations for applications to transition to the cloud, IaaS components, OpenStack fundamentals and more.
While some applications are self-contained and can be easily lifted from an on-premises environment and shifted to a public cloud environment, many applications are more complex and have many more dependencies. In order to transition to a cloud provider, a number of considerations must be taken into account.
- Portability – Applications can be developed and written to make it easier to move to a different computing environment. By limiting dependencies on many other software packages, applications can move from one cloud provider to another. Important considerations for portability include integrating with an operating system (thus, if the OS is portable, so is the application), knowledge of all dependencies, and using well-defined standards.
- Elastic – Many applications can use more than one server to accomplish a task. In the case of HPC-type applications, tens to hundreds of servers might be required. Well-designed applications will scale as the number of servers is increased. One of the considerations in moving to a cloud is whether the cloud provider can accommodate the required number of servers needed to one tenant. More complex is whether the application can scale while it is running, and whether the cloud provider can accommodate such requests.
- IaaS-agnostic – For applications to be able to transition to the cloud, the application must not be tied to any particular piece of hardware, whether it is CPU, accelerator, networking or storage. Although different providers will have different architectures, applications can be designed to make runtime decisions, along with operating system assistance.
IaaS Components
All IaaS providers will have the ability to provide a wide range of hardware for different customer needs. Among these are:
PROCESSORS
While CPUs are fairly standard, the provider should be able to offer a range of instances, or types, of computers for rent. For example, pay more and get better performance, pay more and get more memory; or wait until the provider load is reduced and pay less. NETWORKING A complexity in using the cloud for computations that are dependent on very fast networking is that these fast networks (InfiniBand, for example) may not always be present in the cloud infrastructure. Software applications and the operating system used in the cloud must be able to support a wide range of networking. Depending on the customer’s budget, the networks chosen may not be the most performant.
A critical component of any cloud infrastructure is the storage that will be used for the HPC application.
STORAGE
A critical component of any cloud infrastructure is the storage that will be used for the HPC application. With a wide range of storage technologies to choose from, it is important for the customer to understand the storage architecture that is present in the cloud provider’s infrastructure.
- Near vs. far – Some disk drives (hard disk or solid state storage) will be on the server itself and provide the fastest storage and data access. However, those drives are typically not shared among different servers, making it difficult to share data. Storage area networks contain massive amounts of storage capacity that can be accessed by all of the servers over the network. The combination of fast and intelligent storage running in parallel with the right network can greatly help most HPC applications.
- Parallel file systems – Many HPC applications require the performance of a parallel file system, such as Lustre. Lustre is an open source parallel file system that works with a wide variety of block storage devices, as well as both Ethernet and InfiniBand networking. It is important that the OS fully supports Lustre for those applications that need the highest level of I/O performance.
[clickToTweet tweet=”Many HPC applications require the performance of a parallel file system. #cloudcomputing” quote=”Many HPC applications require the performance of a parallel file system. #cloudcomputing”]
Over the next few weeks this series on the HPC transition to the cloud will cover the following additional topics:
- The HPC Transition to the Cloud
- Cloud Computing
- OpenStack Fundamentals
- Future Directions
You can also download the complete report, “insideHPC Research Report on HPC Moves to the Cloud – What You Need to Know,” courtesy of Red Hat.
