Cloud computing has become another tool for the HPC practitioner. For some organizations, the ability of cloud computing to shift costs from capital to operating expenses is very attractive. Because all cloud solutions require use of the Internet, a basic analysis of data origins and destinations is needed. The analysis is simple and is based on where data are generated. If data are generated in the cloud then using a cloud solution is often beneficial. If on the other hand, data sets are local, then transport to and from the cloud may prove inefficient and slow progress.
For examples, 100 TB of data would require about 3 months to move either to or from the cloud (assuming a fast 100Mbit/sec link). Another consideration is the velocity of data generation. If data are generated at a high velocity it may not be feasible to move to/from the cloud without resorting to physical transport of media.
This the fourth article in a series on InsideHPC’s Guide to Successful Technical Computing.
The effectiveness of the cloud is, therefore, dependent upon the velocity and location of data growth. In general, if the data grow locally, then local processing is usually the best choice.
Finally, data security in the cloud must be considered. If data security is an issue then, public cloud solutions should not be used. Encryption may help ease some concerns, however, it is not possible to process data in the public cloud while it is encrypted.
Cloud Bursting How Much and How Fast
There are two use cases where cloud HPC can be useful even if you have a local HPC asset and data. The first is when part-time capacity is needed. This situation may occur when local resources cannot accommodate high user resource demands over short periods. In this case, it may make sense to have public cloud resources available to meet the temporary needs. Of course, data movement requirements may determine the feasibility of this approach. Another need for cloud computing is the occasion need for capabilities larger than those available locally. Again, this situation is expected to meet a temporary need.
The next article in this series will offer three questions to ensure HPC Success. If you prefer you can download the complete insideHPC Guide to Successful Technical Computing, courtesy of SGI and Intel – Click Here.