insideHPC » inside-BigData

Video: High Performance Computing Trends for 2013

Rich Brueckner — Sun, 16 Jun 2013 13:36:56 +0000

In this video from the HPC Advisory Council European Conference 2013, Addison Snell from Intersect360 Research presents: High Performance Computing Trends for 2013.

This presentation is an overview of the current important trends in HPC, based on the latest end-user research studies and market forecasts. Topics include accelerator adoption, the role of HPC in Big Data, and the ratio of spending between hardware, software, staffing, and facilities.”

Download the slides (PDF).

The post Video: High Performance Computing Trends for 2013 appeared first on insideHPC.

Cray Rolls Out Hadoop Cluster Solution

Rich Brueckner — Thu, 13 Jun 2013 14:54:33 +0000

Today Cray announced a new Hadoop solution that combines supercomputing technologies with an “enterprise-strength” approach to Big Data analytics. Available later this month, Cray cluster supercomputers for Hadoop will pair Cray CS300 systems with the Intel Distribution for Apache Hadoop.

More and more organizations are expanding their usage of Hadoop software beyond just basic storage and reporting. But while they’re developing increasingly complex algorithms and becoming more dependent on getting value out of Hadoop systems, they are also pushing the limits of their architectures,” said Bill Blake, senior vice president and CTO of Cray. “We are combining the supercomputing technologies of the Cray CS300 series with the performance and security of the Intel Distribution to provide customers with a turnkey, reliable Hadoop solution that is purpose-built for high-value Hadoop environments. Organizations can now focus on scaling their use of platform-independent Hadoop software, while gaining the benefits of important underlying architectural advantages from Cray and Intel.”

As you may recall, Cray acquired the CS300 cluster technology from Appro last year. This gives the company a more affordable cluster offering for markets that don’t require Cray’s low latency interconnect technology. Read the Full Story.

The post Cray Rolls Out Hadoop Cluster Solution appeared first on insideHPC.

Hadoop Meets Lustre – Intel Rolls out Big Data Distribution for the Enterprise

Rich Brueckner — Wed, 12 Jun 2013 18:39:00 +0000

Today Intel announced the “first converged HPC and Big Data plaform” with the new Intel Enterprise Edition for Lustre. Paired with Chroma storage management tools from Whamcloud as well as a new adaptor for the Intel Distribution for Apache Hadoop, the new offering provides enterprise-class reliability combined with HPC performance for Big Data applications.

Enterprise users are looking for cost-effective and scalable tools to efficiently manage and quickly access large volumes of data to turn valuable information into actionable insight,” said Boyd Davis, vice president and general manager of Intel’s Datacenter Software Division. “The addition of the Intel Enterprise Edition for Lustre to our big data software portfolio will help make it easier and more affordable for businesses to move, store and process data quickly and efficiently.”

When paired with the Intel Distribution for Apache Hadoop, the Intel Enterprise Edition for Lustre software allows Hadoop to be run on top of Lustre, significantly improving speed in which data can be accessed and analyzed. This allows users to access data files directly from the global file system at faster rates and speeds up analytics time, providing more productive use of storage assets as well as simpler storage management.

The Intel Enterprise Edition for Lustre will be available in early in the third quarter of this year. Read the Full Story.

The post Hadoop Meets Lustre – Intel Rolls out Big Data Distribution for the Enterprise appeared first on insideHPC.

Slidecast: Rogue Wave Software for Developing Parallel, Data-intensive Applications

Rich Brueckner — Wed, 12 Jun 2013 15:46:49 +0000

In this slidecast, Scott Lasica from Rogue Wave Software describes how the company helps its customers meet the challenges of programming at extreme scale.

Developing parallel, data-intensive applications is hard. We make it easier. Rogue Wave works with many scientists performing cutting-edge research and solving Grand Challenge class problems at labs and supercomputer facilities around the world. Time and again, scientists tell us that TotalView provides them with the advanced functionality that makes it possible to quickly fix even complex bugs.”

To learn more about Rogue Wave Software, check out their booth #550 at ISC’13.

Download the MP3 * Subscribe on iTunes* Subscribe to RSS

The post Slidecast: Rogue Wave Software for Developing Parallel, Data-intensive Applications appeared first on insideHPC.

Wired Looks at the NSA’s $2 Billion Datacenter in Utah

Rich Brueckner — Fri, 07 Jun 2013 10:30:54 +0000

Over at Wired Magazine, James Bamford looks at the $2 billion datacenter in Utah that the NSA plans to bring online in September.

Given the facility’s scale and the fact that a terabyte of data can now be stored on a flash drive the size of a man’s pinky, the potential amount of information that could be housed in Bluffdale is truly staggering. But so is the exponential growth in the amount of intelligence data being produced every day by the eavesdropping sensors of the NSA and other intelligence agencies. As a result of this “expanding array of theater airborne and other sensor networks,” as a 2007 Department of Defense report puts it, the Pentagon is attempting to expand its worldwide communications network, known as the Global Information Grid, to handle yottabytes (1024 bytes) of data. (A yottabyte is a septillion bytes—so large that no one has yet coined a term for the next higher magnitude.) It needs that capacity because, according to a recent report by Cisco, global Internet traffic will quadruple from 2010 to 2015, reaching 966 exabytes per year. (A million exabytes equal a yottabyte.)

Read the Full Story.

The post Wired Looks at the NSA’s $2 Billion Datacenter in Utah appeared first on insideHPC.

Podcast: Interview with DDN’s New President, Joe Cowan

Rich Brueckner — Wed, 05 Jun 2013 14:00:18 +0000

In this podcast, Jeff Denworth from DataDirect Networks introduces the company’s new President, Joe Cowan. As a former member of the DDN board, Cowan brings a wealth of experience in the enterprise, a space where DataDirect Networks hopes to grow with its Big Data solutions.

Read the Full Story * Download the MP3 * Subscribe on iTunes * Subscribe to RSS

The post Podcast: Interview with DDN’s New President, Joe Cowan appeared first on insideHPC.

TACC’s Hadoop Cluster Makes Big Data Research More Accessible

Ralph — Mon, 03 Jun 2013 11:00:31 +0000

Over at the Texas Advanced Computing Center, Aaron Dubrow writes that researchers are using a specialized cluster at TACC to do experimental Hadoop-style studies on a current production system.

This system offers researchers a total of 48, eight-processor nodes on TACC’s Longhorn cluster to run Hadoop in a coordinated way with accompanying large-memory processors. A user on the system can request all 48 nodes for a maximum of 96 terabytes (TB) of distributed storage. What’s special about the Longhorn cluster at TACC isn’t simply the beefed-up hardware for running Hadoop; rather it’s the ability for researchers to leverage the vast compute capabilities of the center, including powerful visualization and data analysis systems, to further their investigations. The end-to-end research workflow enabled by TACC could not be done anywhere else, and as a bonus, researchers get access to the full suite of tools available at the center to do computational research.

According to TACC Research Associate Weijia Xu, the best part is that Hadoop is easy to use without requiring users to be experts. It handles a lot of the low-level computing behavior, so people don’t need to have a lot of knowledge about I/O or memory structures to get started.

Read the Full Story.

The post TACC’s Hadoop Cluster Makes Big Data Research More Accessible appeared first on insideHPC.

Video: Science as Voyage – Ian Foster at TEDxCERN

Rich Brueckner — Mon, 27 May 2013 11:30:29 +0000

In this video from the TEDxCERN event, Ian Foster takes us on a journey of Big Process for Big Data, with a call to action for the kind of collaborative processes we need.

The post Video: Science as Voyage – Ian Foster at TEDxCERN appeared first on insideHPC.

Slidecast: Teradata Rolls Out Intelligent Memory Technology

Rich Brueckner — Sat, 18 May 2013 13:43:45 +0000

In this slidecast, Scott Gnau from Teradata Labs presents: Teradata Intelligent Memory.

The introduction of Teradata Intelligent Memory allows our customers to exploit the performance of memory within Teradata Platforms, which extends our leadership position as the best performing data warehouse technology at the most competitive price,” said Scott Gnau, president, Teradata Labs. “Teradata Intelligent Memory technology is built into the data warehouse and customers don’t have to buy a separate appliance. Additionally, Teradata enables its customers to buy and configure the exact amount of in-memory capability needed for critical workloads. It is unnecessary and impractical to keep all data in memory, because all data do not have the same value to justify being placed in expensive memory.”

How does Intelligent Memory work? This animation video does a good job of making this advanced technology look simple.

Read the Full Story * View the slides * Download the MP3 * Subscribe on iTunes * Subscribe to RSS

The post Slidecast: Teradata Rolls Out Intelligent Memory Technology appeared first on insideHPC.

Video: Why the Size of the Data Does Not Define Big Data

Rich Brueckner — Wed, 15 May 2013 17:44:37 +0000

In this video from the 2013 HPC User Forum, John Hengeveld from Intel presents: Big Data Use Cases – The Size of the Data does not define Big Data.

Download the slides (PDF) or check out the HPC User Forum Video Gallery.

The post Video: Why the Size of the Data Does Not Define Big Data appeared first on insideHPC.

A Look at Five HPC Buzzwords

Ralph — Sat, 11 May 2013 14:47:38 +0000

Over at the Adaptive Computing Blog, Ian Nate writes that five HPC buzwords are dominating the conversation as of late.

We’ve noticed a rise in the use of Energy-Efficient Computing, especially when it comes to HPC and Datacenter. A key factor in the future of large-scale HPC systems, energy efficiency is emerging as likely a second big obstacle to reaching exascale. The reason is the cost of powering an exascale system is exponentially higher than the current petascale systems, and power isn’t getting any cheaper. As an industry, HPC will have to weigh the benefits, or need, for exascale, versus the cost to house and power such systems. The good news is that many systems in Europe are already thinking green because of the higher energy costs. And, we’re seeing a stronger presence for Green Computing in the United States, with systems like NICS’ Beacon reaching the top of the Green500, a list that has picked up significant steam since its initial release in 2007.

The other buzzwords include Big Data, Exascale, Petascale Race, and HPC Cloud. Read the Full Story.

The post A Look at Five HPC Buzzwords appeared first on insideHPC.

Radio Free HPC Fireside Chat – HPC Embraces Big Data

Rich Brueckner — Wed, 08 May 2013 14:01:55 +0000

In this slidecast, the Radio Free HPC team interviews Fritz Ferstl, CTO of Univa. Topics include Big Data, HPC, and the continuing convergence of both.

While what we think of as traditional HPC may differ greatly from Big Data analytics, that seems to be changing. With a long history in high performance computing and customers in both worlds, Ferstl shares his unique perspective on where the two worlds overlap and where the potential is greatest for synergy in the future.

This has to be our best show yet, so be sure to check it out.

View the slides on Slideshare * Download the MP3 * Download the mobile video * Download 1024p Video * Subscribe on iTunes * RSS Feed

The post Radio Free HPC Fireside Chat – HPC Embraces Big Data appeared first on insideHPC.

Green Graph 500 Launches to Boost Energy Efficient Big Data Computing

Staff — Tue, 07 May 2013 16:37:39 +0000

In this special guest feature, Torsten Hoefler from ETH Zurich writes that the new Green Graph500 aims to boost energy-efficient Big Data Computing.

“Big Data” can be analyzed in various ways. The most successful and prevalent programming model, MapReduce, convinces by its flexibility toadapt to hardware performance variations and faults. However, even though MapReduce covers a huge majority of use-cases, it has its limits for graph computations. Complex graph algorithms become more important as our analysis capabilities grow. For example, problems such as finding hubs in social network graphs are routinely answered today. The underlying algorithm, betweenness centrality, utilizes a graph traversal similar to breadth first search or shortest path search. Systems such as Google’s Pregal, Apache’s Giraph, the (Parallel) Boost Graph Library, and Stanford’s GPS are just some examples for emerging frameworks to handle large-scale graph computations. In order to efficiently compare architectures and possibly programming frameworks, the Graph 500 benchmark strives to establish a database for performance of a standardized breadth first search on various platforms.

As energy is becoming a bigger concern than hardware purchasing costs in large-scale data centers and supercomputing centers, it becomes mandatory to not only consider the performance of such computations but also their exact energy consumption. In fact, if the current cost trends continue, then energy consumption will soon be more important than absolute performance. Such discussions are highly relevant for operators of large data centers such as Google, Amazon, and Yahoo, as well as large supercomputing centers operated by the DOE (e.g., LLNL, Sandia,LANL, ORNL) and the NSF (e.g., NCSA, SDSC, PSC). We are thus looking forward to interesting future developments targeting exascale as well as Big Data architectures and programming frameworks.

We introduce the Green Graph 500 list which fulfills a variety of purposes. First and foremost it is to establish the practice to compete not only for the highest performance but also for the highest energy efficiency, directly benefiting society. It is also set out to collect historical data about developments that may allow us to predict future trends very similar to what the top 500 list has achieved in the past(who doesn’t like to put up a top 500 slide to project out FLOP rate for the next 10 years?). The list will also allow us to compare the energy efficiency of a specific computer for certain tasks, e.g.,dense linear algebra (a problem mainly limited by memory size and CPU peak floating point performance) versus graph search (a problem mainly limited by memory access rates and global system bandwidth). Those two metrics together may serve as a measure to generate more efficient balanced systems as well as special-purpose systems for one of those tasks.

Finally, the new Green Graph 500 list is not meant to compete with any of the existing lists. It is indeed complementary, filling an important gap in the field. In fact, the rules are designed to be similar to the established Green 500 rules (similar, not identical, for example with regards to the network) so that comparisons can easily be made in the future. It also directly integrates with the Graph 500 list and submission system to guarantee one-to-one comparisons (a submission record may be in the Green Graph 500 as well as the Graph 500 even though the lists are ranked by different indices).

The Green Graph 500 list is soliciting submissions from everyone through the Graph 500 submission system. To submit to the list, simply start a normal Graph 500submission and select “Submit to Green Graph 500″ or “Submit to both lists”. The only additional data you need for a Green Graph 500submission is the actual power draw of your system during the benchmark.

Another small difference between Graph 500 and it’s Green peer is the measurement methodology. Since most power meters are not accurate enough to measure the rather short actual BFS run (not including the post-check etc.), we offer a slightly modified version of the reference benchmark which allows to run the BFS in a tight loop long enough for a low-time resolution energy meter to measure the exact energy consumption. This benchmark will also report a Graph 500 number valid for submission. For runs with a custom implementation, this would need to be ensured manually (4-5 lines of C Code suffice for this). The submission opens together with the official Graph 500 submission.

As a sneak peek, we prepared a sample list from March 2013′s energy submissions (which may not have followed all the official rules, thus, the list is not official).

The Green Graph 500 list is maintained by Torsten Hoefler from ETH Zurich in collaboration with the Graph 500 executive committee. For questions or comments please contact info@green.graph500.org.

The post Green Graph 500 Launches to Boost Energy Efficient Big Data Computing appeared first on insideHPC.

Indiana University Helps NASA Manage Big Data for Operation IceBridge

Rich Brueckner — Fri, 03 May 2013 11:00:00 +0000

Indiana University has contributed Big Data expertise and infrastructure to NASA’s Operation IceBridge, a decade-long polar ice monitoring project.

For the past four years, IU Research Technologies, a cyberinfrastructure and service center affiliated with the Pervasive Technology Institute (PTI), has provided IT support for the Center for Remote Sensing of Ice Sheets (CReSIS), a National Science Foundation Science and Technology Center led by the University of Kansas. Kansas scientists provide NASA with the radar technology that measures the physical interactions of polar ice sheets in Greenland, Chile and Antarctica. IU experts bring innovative data management and storage solutions to the missions.

Essentially, IU has built a supercomputer that can fly,” said Rich Knepper, manager of IU’s campus bridging and research infrastructure team within Research Technologies. “During this current mission, our system provided analysis of radar data as the data was collected – in real time — allowing mission scientists to see the ice bed information as the plane flies over the Arctic.”

Read the Full Story.

The post Indiana University Helps NASA Manage Big Data for Operation IceBridge appeared first on insideHPC.

High Performance RDMA-based Design for Big Data and Web 2.0 memcached

Rich Brueckner — Fri, 26 Apr 2013 11:30:28 +0000

In this video from the 2013 Open Fabrics Developer Workshop, D.K. Panda from Ohio State University presents: High Performance RDMA-based Design for Big Data and Web 2.0 memcached.

Check out more presentation videos at our Open Fabrics Workshop Video Gallery. Most of the slides from the Workshop have been posted as well.

The post High Performance RDMA-based Design for Big Data and Web 2.0 memcached appeared first on insideHPC.

Video: Accelerating Big Data over RDMA

Rich Brueckner — Thu, 25 Apr 2013 13:37:20 +0000

In this video from the 2013 Open Fabrics Developer Workshop, Sreev Doddabalapur from Mellanox presents: Accelerating Big Data over RDMA.

You can check out more OFA videos at our Open Fabrics Workshop Video Gallery.

The post Video: Accelerating Big Data over RDMA appeared first on insideHPC.

SGI Infinite Storage Gateway Appliance Implements DMF in Minutes

Rich Brueckner — Mon, 22 Apr 2013 15:23:06 +0000

Today SGI announced the InfiniteStorage Gateway, an appliance that delivers virtualized data management to lower the cost of high volume storage. Why an appliance? SGI’s Floyd Christofferson described it as a way to “install DMF within minutes and enable greatly simplified storage management for Big Data.”

The SGI InfiniteStorage Gateway reduces the dependency on high-cost primary storage by creating a virtualized storage fabric that can include any mixture of disk, tape, Zero-Watt Disk or MAID, and object storage. While appearing to users and applications simply as online data, SGI InfiniteStorage Gateway offers IT administrators the ability to keep data protected and online at a fraction of the cost of primary storage systems.

As data growth has continued to sky-rocket, IT organizations increasingly face the problem of infrastructure fragmentation, and the fact that their most expensive primary storage arrays are often used to house mostly inactive data,” said Laura DuBois, program vice president, IDC Storage Systems, Software and Solutions. “Data management is not only about the performance of active data today. It also must provide a seamless long-term strategy for all data that keeps costs at a minimum and reduces IT administrative burden without impacting users.”

With up to 276TB of onboard capacity in a single 4U appliance, the gateway automatically places data on any or all storage devices and locations based upon what works best for the access requirements and data protection policies.

Read the Full Story or View the Slides on Slideshare.

The post SGI Infinite Storage Gateway Appliance Implements DMF in Minutes appeared first on insideHPC.

Canada’s Top Supercomputing Conference Comes to Ottawa June 2-6

Staff — Wed, 17 Apr 2013 16:13:48 +0000

The 2013 High Performance Computing Symposium is open for registration and technical program submissions. Canada’s top supercomputing conference is being held at the University of Ottawa from 2-6 June.

HPCS 2013 will address the role of ‘big data’ and ‘big computing’ – using the largest, most powerful computers available – for breakthroughs and innovation in medicine, industry, government and academia including:

The role of Big Data in brain research and transplant surgery;
How Big Data and supercomputing saves time and resources in mining and exploration;
Challenges governments face with massive quantities of data and powerful computing in everything from privacy to security of critical infrastructure such as power grids; and
How Canadian business and industry can achieve the height of global competitiveness through the power of data analysis and supercomputing.

Big Data and high-performance computing is playing a bigger role every day in our lives and in our workplaces,’ said Bill Appelbe, president and CEO of Compute Canada. “The High Performance Computing Symposium brings together some of the world’s leading visionaries, researchers, entrepreneurs and users of Big Data and big computing to learn of the latest breakthroughs and future applications.”

This story appears here as part of a cross-publishing agreement with Scientific Computing World.

The post Canada’s Top Supercomputing Conference Comes to Ottawa June 2-6 appeared first on insideHPC.

Obama Initiative Leverages Big Data to Explore the Brain

John Kirkley — Sat, 06 Apr 2013 15:07:43 +0000

This week President Obama announced a research initiative that has the ambitious goal of “revolutionizing our understanding of the human brain,” according to a White House press release.

Know as BRAIN (Brain Research through Advancing Innovative Neurotechnologies), the initiative is being launched in FY 2014 with an initial budget of about $100 million, a modest amount given the project’s goals.

In short, BRAIN is designed to help researchers find “…new ways to treat, cure, and even prevent brain disorders, such as Alzheimer’s disease, epilepsy, and traumatic brain injury.” Included is support for new technologies that will allow researchers to produce dynamic pictures of the brain that show how individual brain cells and complex neural circuits interact in real time.

This is a foray into Big Data. The initiative will let researchers amass and analyze the data needed to “…explore how the brain records, processes, uses, stores, and retrieves vast quantities of information, and shed light on the complex links between brain function and behavior.”

Among the many public and private organizations involved in the effort are the National Institutes of Health (NIH), the Defense Advanced Research Projects Agency (DARPA), and the National Science Foundation (NSF). NSF in particular is leading the charge in applying the technologies and techniques of Big Data to the initiative.

The National Science Foundation will play an important role in the BRAIN Initiative because of its ability to support research that spans biology, the physical sciences, engineering, computer science, and the social and behavioral sciences,” according to the White House release. “The National Science Foundation intends to support approximately $20 million in FY 2014 in research that will advance this initiative, such as the development of molecular-scale probes that can sense and record the activity of neural networks; advances in ‘Big Data’ that are necessary to analyze the huge amounts of information that will be generated, and increased understanding of how thoughts, emotions, actions, and memories are represented in the brain.”

In a story in Information Week posted the same day, senior editor J. Nicholas Hoover, writes, “On a conference call with reporters after the President’s announcement, National Institutes of Health director Francis Collins said that the brain-mapping initiative might eventually require the handling of yottabytes of data. A yottabyte is equal to a billion petabytes.”

That’s Big Data at its mind-boggling best.

Read the Full Story.

The post Obama Initiative Leverages Big Data to Explore the Brain appeared first on insideHPC.

DSSD is Andy Bechtolsheim’s Secret Chip Startup for Big Data

Rich Brueckner — Thu, 04 Apr 2013 22:12:17 +0000

Over at GigaOm, GigaStacey writes that the solution for better and faster storage may lie in DSSD, a stealthy chip startup backed by Andy Bechtolsheim. Founded in 2010 by Sun Alums Jeff Bonwick and Bill Moore, DSSD is trying to build a chip that would improve the performance and reliability of flash memory for high performance computing, newer data analytics, and networking.

My sources tell me the startup is building a new type of chip — they said it’s really a module, not a chip — that combines a small amount of processing power with a lot of densely-packed memory. The module runs a pared-down version of Linux designed for storing information on flash memory, and is aimed at big data and other workloads where reading and writing information to disk bogs down the application. This fits with the expertise of the team, but this is a problem that others are trying to solve as well with faster and cheaper SSDs and targeted software to to optimize the flow of bits to a database. But the proposal here appears to be about designing an operating system that takes advantage of the difference in Flash memory when compared to hard drives to boost I/O.

Read the Full Story.

The post DSSD is Andy Bechtolsheim’s Secret Chip Startup for Big Data appeared first on insideHPC.