In this video from the HPC Advisory Council Switzerland Conference, Derek Burke from Panasas presents: Introducing Panasas ActiveStor 14.
Video: GPFS CACHE over 200 Km
In this video from the HPC Advisory Council Switzerland Conference, Stefano Claudio Gorini from the Swiss Supercomputing Centre Lugano-Zurich presents: GPFS CACHE over 200 Km.
25th Anniversary of RAID is a Time to Clear the Air
Where would we be without RAID? Over at the Panasas Blog, Geoffrey Noer writes that the 25th anniversary of the landmark “Berkeley RAID Paper” is an opportune time to clear up misconceptions about erasure codes and the current state of RAID.
According to Noer, erasure codes aren’t “what comes next” after RAID; they’re inextricably linked. In fact, he argues that erasure codes should still be considered “RAID,” particularly if one agrees with the Berkeley paper that the concept of RAID applies to software-based protection of files as well as hardware-based device protection.
The second part of the confusion appears to be that some people consider RAID to be a concept that only applies to the protection of entire physical devices (i.e. hard drives). While the original RAID paper does not cover per-file RAID, it does explicitly state that although the examples in the paper assume a hardware-based approach, implementing the algorithms in software could be a superior approach depending on the circumstances. Fast forward twenty-five years and this is precisely what has happened. Software-based approaches are proving to be architecturally more flexible, enabling highly scalable RAID implementations. This is ideally achieved by applying RAID algorithms on a per-file basis, employing erasure codes to protect against physical device failure by spreading fragments of files across storage hardware elements. At Panasas, we believe this should still be called RAID as the purpose has not changed (protect against failures) and RAID algorithms (erasure codes) are still largely the same ones being applied to protect data. The difference is that the RAID algorithms are being more cleverly applied by performing them in software, on a per-file basis.
Read the Full Story.
Blue Waters Ready to Handle Floods of Data
Big Data requires big computing, and the University of Illinois at Urbana-Champaign is doing its part with the launch of Blue Waters, one of the world’s fastest supercomputers.
U of I held an open house a couple of weeks ago, inviting one and all to visit its National Petascale Computing Facility and kick the tires on the $200 million machine built by Cray and funded by the National Science Foundation.
This is a petaflop machine designed to handle the challenging Big Data requirements associated with a wide range of problems – everything from unraveling complex biological systems to simulating the evolution of the cosmos.
This is where you go to get answers to questions about how the world works,’ says Bill Gropp, a computer science professor and one of four U of I researchers who oversaw the five-year development of the machine,” according to a story in Crain’s Chicago Business. The article goes on to say, “Blue Waters will keep the university in the lead on large-scale computing as researchers from around the country apply to the National Science Foundation to use the machine to crunch data for medical research, astrophysics, aerodynamics, weather forecasting, national security and other uses.”
This is not your everyday supercomputer. The Blue Waters system is a Cray XE/XK hybrid machine made up of AMD 6276 “interlagos” processors with a nominal clock speed of at least 2.3 GHz) and NIVIDIA GK110 Kepler accelerators, all connected by the Cray Gemini torus interconnect.
Blue Waters is capable of a sustained speed of over one petaflop, allowing it to perform more than one quadrillion calculations per second. The water-cooled system is housed in 276 black cabinets topped by silvery coolant pipes.
In addition to being really fast, Blue Waters has more than enough memory to handle Big Data requirements – 1.5 petabytes of total system memory and 300 petabytes of long-term storage.
In the Crain’s article, Gropp is quoted as saying, “We want people to ask, ‘What could you do if you could put massive amounts of data on a system and access it in microseconds?’”
The short answer is, “More than you can ever imagine.”
Read the Full Story.
Video: Architecting High Availability Lustre Storage Solution – ClusterStor 6000
In this video from the HPC Advisory Council Switzerland Conference, Torben Kling Petersen from Xyratex presents: Architecting High Availability Lustre Storage Solution – ClusterStor 6000.
Part of the ClusterStor family, ClusterStor 6000 is designed to support installations with linear performance scalability in less space, scaling from up to 6 gigabytes per second to installations providing 1 terabyte per second file system throughput, as well as linear data storage capacity from terabytes up to tens of petabytes.
Video: Dell’s Modular HPC – Exascale Block by Block
In this video from the HPC Advisory Council Switzerland Workshop, Kris Buggenhout from Dell presents: Modular HPC, Exascale Block by Block.
Video: A Generic Methodology for Optimizing an HPC Storage System
In this video from the HPC Advisory Council Switzerland Conferenc, Zhiqi Tao from Intel (formerly of Whamcloud) presents: A Generic Methodology for Optimizing an HPC Storage System.
Designing a large scale, high performance storage system presents significant challenges. This paper describes a step-by-step approach to designing a storage system and presents a design methodology based on an iterative approach that applies at both the component level and the overall system level. The paper includes a detailed case study in which a Lustre storage system is designed using the approach and methodology presented.
Read the Whamcloud Whitepaper on this topic or Download the Slides (PDF).
In related news, the Lustre User Group conference will take place in San Diego April 16-18.
Video: NetApp – Scaling Storage from Seismic Depths to Sequoia Heights
In this video from the HPC Advisory Council Switzerland Conference, Alexander Sammer from NetApp presents: Scaling Storage from Seismic Depths to Sequoia Heights. Download the slides (PDF).
Video: The Future of Network-based Storage
In this video from the HPC Advisory Council Switzerland Conference, Brent Gorda from Intel presents: The Future of Network-based Storage. Gorda joined Intel in July 2012 as part of Intel’s acquisition of Whamcloud. Since then, Gorda’s team has continued to work on Lustre as well as conduct R&D on Darpa’s Fast Forward Storage & IO program.
In related news, the Lustre User Group conference will take place in San Diego April 16-18.
Think of This: Most of the World’s Data is Unanalyzed
The idea that we use only 10 percent or less of our brain is one of those persistent myths that stubbornly refuses to go away. In reality, that bit of gelatinous grey matter between our ears has been extensively mapped and it appears that most of it has a function. So the notion that if only we could harness that underused 90 percent of our brain we could develop superhuman mental abilities, needs to be relegated to the urban legend scrap heap.
But what about the idea that the world of Big Data is in even worse shape – that only one percent of what’s actually out there is being analyzed?
That’s the contention of Gurjeet Singh, a guest contributor to a recent issue of GigaOM.
Singh is the co-founder and CEO of Ayasdi, a discovery platform built on topological data analysis technology. He says that to make use of the untouched 99 percent of available data – some one quintillion bytes that are collected every day (or, according to IBM, a daily harvest of 2.5 quintillion bytes) – we have to “fundamentally change the way we gain knowledge from data.”
Singh calls on the next wave of Big Data solutions to: empower domain experts such as biologists, geologists, etc. – not just the data scientists; accelerate the pace of discovery so we can get insights faster; create new levels of machine intelligence – human thought is too slow for many Big Data operations; and design systems that are able to efficiently analyze both structured and unstructured data, especially the latter in all its myriad forms.
When it comes to the evolution of big data, we’ve only begun to scratch the surface,” Singh writes. “It stands to reason that if we continue to analyze 1 percent of data, then we’ll only tap into 1 percent of its potential. If we’re able to analyze the other 99 percent, then think about all of the ways that we can change the world. We can accelerate economic growth, cure cancer and other diseases, reduce the risk of terrorist attacks, and many other big ticket challenges that we’re faced with. That’s something that we can all rally around.”
Read the Full Story.
The Villany of Silent Data Corruption
Over at the Emulex Blog, Sonny Singh writes that increasing complexity of data center environments and growth in storage have led to significant concerns about silent data corruption.
But really what it comes down to is without end-to-end protection technology, data corruption can go unnoticed until recovery is difficult and costly or even impossible to perform. Furthermore, without end-to-end integrity checking, these silent data corruptions can lead to unexpected and unexplained problems.
Read the Full Story or check out the podcast on this topic from Radio Free HPC.
NetApp E-5500 Storage System Delivers Record Performance per Spindle
Today NetApp announced the new NetApp E5500 storage system for Big Data and HPC applications. Derived from Engenio technology acquired from LSI, the seventh-generation E-Series is a high-density platform with record per-spindle throughput.
The SGI InfiniteStorage 5600, which is an OEM version of the NetApp E5500, has produced a new SPC-2 result confirming the performance and cost efficiency of the new E5500; it showcases the performance possibilities the E5500 unlocks for HPC and big data organizations. The audited, peer-reviewed SPC-2 result demonstrates the highest throughput per spindle by more than 2.5 times over the nearest non-NetApp published result.
Read the Full Story.
In related news, SGI rolled out the InfiniteStorage 5600 storage system today, noting that the flexibility of the platform enables users to push the limit without breaking their budget.
1 Terabyte/sec File Systems Enable Big Fast Data
Over at the new Cray Blog, VP of Storage Barry Bolding writes that Big Data at the company is synonymous with very fast data access. As evidenced by the Lustre-powered Terabyte/sec file system on Blue Waters, the time for Big Fast Data has come.
Success was achieved on Oct. 7, 2012 when Cray and NCSA measured 1.037 TB/sec of performance to a single Lustre filesystem and 1.137 TB/sec of simultaneously to all three Lustre filesystems. To put this achievement into mass media perspective, 1TB/sec is equivalent to downloading about 125 copies of Dead Space 3 per second, or about 250,000 songs through Spotify per second. This is a milestone for parallel filesystems, a milestone for Lustre, a milestone for Cray and a milestone for our customer at NCSA. Best of all this the technology is being put to good use by a cadre of National Science Foundation users in solving some of the most challenging scientific problems facing our world today.
Read the Full Story.
Violin Memory and Toshiba Team up for Mass Adoption of Memory Infrastructure
In this slidecast, Narayan Venkat from Violin Memory describes how the company’s new alliance with Toshiba will help foster a whole new world of applications that perform at the speed of memory.
Our new focus on PCIe cards will allow both companies to drive radical new economics that lead to the mass adoption of memory-based architectures,” said Don Basile, CEO of Violin Memory. “NAND memory is now a requirement at every level from the smart connected device to the core of the cloud and the enterprise data center. Violin’s combined portfolios continue our leadership across the evolving memory-based solution market.”
Read the Full Story * Download the MP3 * Download the Slides * Subscribe on iTunes
RCE Podcast Looks at GlusterFS File System
In a new RCE Podcast, Brock Palen and Jeff Squyres speak with Jeff Darcy of Red Hat about Gluster FS, a distributed scalable open source filesystem.
Jeff has been working on distributed storage since NFS version 2 at Encore in 1990. Most recently he worked on Lustre at SiCortex, and then started his own project HekaFS at Red Hat. Since Red Hat acquired Gluster, he has been an architect and ambassador for that project.
