Satoshi Matsuoka Presents: A Look at Big Data in HPC

satoshi

In this video from the DDN User Group at ISC’14, Satoshi Matsuoka from the Tokyo Institute of Technology presents: A Look at Big Data in HPC. “HPC has been dealing with big data for all of its existence. But it turns out that the recent commercial emphasis on big data, has coincided with a fundamental change in the sciences as well. As scientific instruments and facilities produce large amounts of data in an unprecedented rate, the HPC community is reacting to this, with revisiting architecture, tools, and services to address this growth in data.”

Catalyst Supercomputer at Livermore Open for Big Data Innovation

Catalyst2

“The increased storage capacity of the system (in both volatile and nonvolatile memory) represents the major departure from classic simulation-based computing architectures common at DOE laboratories and opens new opportunities for exploring the potential of combining floating point focused capability with data analysis in one environment. The machine’s expanded DRAM and fast, persistent NVRAM are well suited to a broad range of big data problems including bioinformatics, business analytics, machine learning and natural language processing.”

Marc Hamilton Looks at China HPC

“Like the US, Japan, and Europe, China still has plans to build giant HPC systems like Tianhe. However, increasingly these systems are being looked at to support commercial HPC workloads like machine vision in a cloud environment in addition to just scientific data processing.”

The Scary Side of AI and Big Data

1c9b78d

Douglas Eadline writes that recent big investments in AI technology by IBM and Google show that intelligent systems are the future of big business. The problem is, these advancements could come at the expense of our privacy.

Porting Hadoop to HPC

intel

Ralph H. Castain from Intel presented this talk at the Adaptive Computing booth at SC13. “The solution allows customers to leverage both their HPC and big data investments in a single platform, as opposed to operating them in siloed environments. The convergence between big data and HPC environments will only grow stronger as organizations demand data processing models capable of extracting the results required to make data-driven decisions.”

How CRISP is Tackling the Data Deluge in International Science

csm_2011-11-16_0017_01_4b8e43e445

“CRISP is helping to standardize data-access through a common identity system, or federated identity, allowing identities to transcend facilities or countries, and providing scientists with a single online identity or login to access all their data in one place – regardless of where it’s from.”

Slidecast: Introducing the Terascala Intelligent Storage Bridge

Slide1

“HPC workflows are most efficient when researchers have the data they need, when the need it. The Intelligent Storage Bridge acts like an air traffic controller, automatically moving large data sets on and of scratch using the maximum available network, compute, and storage resources.”

New Moab Task Manager & Support for Intel HPC Distribution for Hadoop

robclyde

New innovations from Adaptive Computing include: Moab Task Manager, a localized decision-making tool within Moab’s HPC Suite that enables high-speed throughput on short computing jobs. Adaptive has also announced a partnership with Intel to integrate Moab/TORQUE workload management software with the Intel HPC Distribution for Apache Hadoop software, which combines the Intel Distribution for Apache Hadoop software with the Intel Enterprise Edition of Lustre software.

Panel Discussion: Solving the HPC Data Deluge

bigdatapanel

As science drives a rapidly growing need for storage, existing environments face increasing pressure to expand capabilities while controlling costs. Many researchers, scientists and engineers find that they are outgrowing their current system, but fear their organizations may be too small to cover the cost and support needed for more storage. Join these experts for a lively discussion on how you can take control and solve the HPC data deluge.

Creating a Better Infrastructure to Manage Big Data

Trev Harmon

Over the course of this talk, Trev Harmon looks back to the utility computing vision of Douglas Parkhill and proposes an application-centric workflow for the future that fulfills that vision across many disciplines of computing.