Sign up for our newsletter and get the latest big data news and analysis.

Carr on Mathematica's HPC integration

We pointed to this back in early November when it was announced. Nick Carr’s article provides a few more details on the specifics of how the integration between Mathematica and Amazon’s compute resources happens

The workflow is very simple to understand and it takes very few clicks to deploy your code in the cloud. A typical Mathematica user develops code in their standard notebook interface, a programming concept that defines their input code and output results, including graphics. The user specifies input cells, output cells and other parameters. Mathematica will evaluate one input cell at a time so evaluation could take a lot of time to process on one machine. Now, with the new Cloud service, users can evaluate the entire notebook in one shot by pushing it to the cloud.

The HPC Cloud Service lets users take the entire notebook, click a few buttons in the HPC Cloud Service GUI and ask it to run it in the cloud. The HPC Cloud Service evaluates the code, runs it in parallel Mathematica sessions, bundles up the results and notifies the user. In other words, a user can test the code (a Mathematica Notebook) with a small amount of input and then increase size of the input to a more realistic size, push it to the cloud so it runs on hundreds and even thousands of nodes in parallel, and get notified when its done.

Many research labs have been building software like this for the past few years as one offs, and I’ve even been part of an effort to create a toolset to do this for the HPC assets run by the DOD’s HPC Modernization Program (it’s now deployed enterprise-wide within the program). But all of these efforts have very limited scope. Carr is right to point out that this is a real step for the democratization of HPC out of the sandbox of the small community of federal researchers doing it previously.

Resource Links: