In this video, Dan Reed from the University of Iowa describes the era of Exascale Computing and Big Data. In a recent paper co-authored with Jack Dongarra, Reed makes an impassioned plea for hardware and software integration and cultural convergence.
The possibilities for this convergence are legion. The algorithms underlying deep machine learning would benefit from the parallelization and data movement minimization techniques commonly used in HPC applications and libraries. Similarly, the approaches to failure tolerance and systemic resilience common in cloud software have broad applicability to high-performance computing. Both domains face growing energy constraints on the maximum size of feasible systems, necessitating shared focus on domain-specific architectural optimizations that maximize operations per joule.