In this video from the 2013 Open Fabrics Developer Workshop, Doug Oucharek from Intel presents an Update on Lustre, OpenSFS, and FastForward InfiniBand.
Download the slides (PDF). You can check out more OFA videos at our Open Fabrics Workshop Video Gallery.
In related news, slides from LUG 2013 are now posted.
In this video from the Lustre User Group 2013 conference, Josh Judd from Warp Mechanics presents: Warp Mechanics ZFS Array.
The WARP Mechanics 39830 is a turnkey network-attached non-volatile RAM + SSD system with industry-leading price, performance, and scalability. This system maximizes the IOPs performance for the most demanding application profiles. It is an ultra-dense space and power saving solution. This is optimal for large-scale IO intensive workloads with large live data sets. The 50x high capacity 2TB SSD modules per 4U enclosure are configured into five 10-disk RAID 6 sets to maximize protection and performance. Each RAID set has a two NV-RAM modules serving as write cache. These RAID sets are added to the overall ZFS storage pool and can be allocated to a nearly limitless number of any sized volumes presented to hosts. This yields a flexible 100TB of usable RAID protected SSD storage.
Download the slides (PDF). Check out more Lustre presentation videos at our LUG 2013 Video Gallery.
In this video from the Lustre User Group 2013 conference, Eric Barton from Intel presents: Lustre – Fast Forward to Exascale.
Back in July 2012, Whamcloud was awarded the Storage and I/O Research & Development subcontract for the Department of Energy’s FastForward program. Shortly afterward, the company was acquired by Intel. The two-year contract scope includes key R&D necessary for a new object storage paradigm for HPC exascale computing, and the developed technology will also address next-generation storage mechanisms required by the Big Data market.
The subcontract incorporates application I/O expertise from the HDF Group, system I/O and I/O aggregation expertise from EMC Corporation, object storage expertise from DDN, and scale testing facilities from Cray, teamed with file system, architecture, and project management skills from Whamcloud. All components developed in the project will be open sourced and benefit the entire Lustre community.
Download the slides (PDF). Check out more presentations at our LUG 2013 Video Gallery.
Today SGI announced the InfiniteStorage Gateway, an appliance that delivers virtualized data management to lower the cost of high volume storage. Why an appliance? SGI’s Floyd Christofferson described it as a way to “install DMF within minutes and enable greatly simplified storage management for Big Data.”
The SGI InfiniteStorage Gateway reduces the dependency on high-cost primary storage by creating a virtualized storage fabric that can include any mixture of disk, tape, Zero-Watt Disk or MAID, and object storage. While appearing to users and applications simply as online data, SGI InfiniteStorage Gateway offers IT administrators the ability to keep data protected and online at a fraction of the cost of primary storage systems.
As data growth has continued to sky-rocket, IT organizations increasingly face the problem of infrastructure fragmentation, and the fact that their most expensive primary storage arrays are often used to house mostly inactive data,” said Laura DuBois, program vice president, IDC Storage Systems, Software and Solutions. “Data management is not only about the performance of active data today. It also must provide a seamless long-term strategy for all data that keeps costs at a minimum and reduces IT administrative burden without impacting users.”
With up to 276TB of onboard capacity in a single 4U appliance, the gateway automatically places data on any or all storage devices and locations based upon what works best for the access requirements and data protection policies.
Read the Full Story or View the Slides on Slideshare.
In this video from LUG 2013, Jeff Johnson from Startup Aeon Computing presents an overview of the company’s innovative Lustre storage solutions.
There are many storage solutions available in the market but not all of them do Lustre well. We set out to design a Lustre platform that was good at Lustre data and I/O profiles. Part of that design, in addition to performance, is that it follows Aeon Computing’s business philosophy in that there is no unnecessary, extraneous bull___t that gets in the way.
Download the slides (PDF). Read the Full Story or check out more presentations at our LUG 2013 Video Gallery.
In this video from the Lustre User Group 2013 conference, Christopher Morrone from Lawrence Livermore National Laboratory presents: Sequoia and the ZFS OSD.
Download the slides (PDF). Check out more presentations at our LUG 2013 Video Gallery.
In this video from the Lustre User Group 2013 conference, Torben Kling Petersen from Xyratex presents: The Lustre Community in Europe.
Download the slides (PDF). Check out more presentations at our LUG 2013 Video Gallery.
In this video from LUG 2013, Dave Fellinger from DDN presents: The State of the Lustre File System and The Lustre Development Ecosystem: A 2013 Report Card.
Download the slides (PDF). Check out more presentations at our LUG 2013 Video Gallery.
Today DDN announced that Oak Ridge National Laboratory has selected the company to build the “world’s fastest storage system” to power the #1 Titan supercomputer.
Using DDN’s SFA12K-40 storage systems as the backbone for Spider II, this new file storage system is designed with 40 petabytes of raw capacity and is capable of ingesting, storing, processing and distributing research data at unprecedented speed. This amount of storage capacity is equivalent to more than 227,000 miles of stacked books – or the distance from ORNL’s facility in Oak Ridge, TN to the moon – and enables ORNL to dramatically increase Titan’s computational efficiency and deliver vastly more accurate predictive models than ever before.
The DDN storage system will deliver over 1 Terabyte/sec in throughput to drive radical advances in science and Big Data analysis essential to DOE and Office of Science Missions.
When building the world’s fastest system for data intensive computing, we carefully considered all aspects of high-throughput I/O infrastructure and how efficient storage platforms can complement our supercomputer’s efficiency. The ORNL and DDN teams have worked together to architect a file system designed to enhance the performance of our Titan supercomputer and enable our users to achieve unprecedented simulations and big data insights through massively scalable computing.”
Read the Full Story.
Over at Nex7′s Blog, Andrew Galloway from Nexenta Systems writes that while ZFS is one of the most powerful, flexible, and robust filesystems, it does have its own share of caveats, gotchya’s, and hidden “features.”
Deduplication Is Not Free. Another common misunderstanding is that ZFS deduplication, since its inclusion, is a nice, free feature you can enable to hopefully gain space savings on your ZFS filesystems/zvols/zpools. Nothing could be farther from the truth. Unlike a number of other deduplication implementations, ZFS deduplication is on-the-fly as data is read and written. This creates a number of architectural challenges that the ZFS team had to conquer, and the methods by which this was achieved lead to a significant and sometimes unexpectedly high RAM requirement. Every block of data in a dedup’ed filesystem can end up having an entry in a database known as the DDT (DeDupe Table). DDT entries need RAM. It is not uncommon for DDT’s to grow to sizes larger than available RAM on zpools that aren’t even that large (couple of TB’s). If the hits against the DDT aren’t being serviced primarily from RAM or fast SSD, performance quickly drops to abysmal levels. Because enabling/disabling deduplication within ZFS doesn’t actually do anything to data already on disk, do not enable deduplication without a full understanding of its requirements and architecture first. You will be hard-pressed to get rid of it later.
Read the Full Story.
Over at the Spectra Blog, Molly Rector writes that several trends came out loud and clear at the recent Storage Networking World conference.
The education track was excellent. I spent a majority of the three days attending the sessions, most of which were full or close to full throughout the day. From these sessions and conversations with attendees, a few topics and trends emerged including:
- Cloud Interface Technologies. Evolving cloud interface technologies that allow movement of data between clouds.
- SSD Hybrid Systems. SSD market is needing a few SSD-only appliances, and the true differentiation and larger demand for SSD-storage is in hybrid systems.
- Data Growth. How to architect to deal with rapid unstructured data growth.
Molly also points us to this video from the conference, which is a parody of Adele’s Rolling in the Deep. Backup has never been so catchy.
Hierarchical storage management is not new to the HPC crowd, but the idea of optimizing NAS may just be a new concept to many. This week Avere Systems announced that the company’s new FXT 3800 hybrid storage appliance can now automatically tier data across four media types: RAM, SSD, SAS and SATA HDDs, delivering maximum performance for the hottest files. At the same time, the device moves “cold” data out of the performance tier and onto SATA to minimize costs and shrink the data storage footprint.
The performance gains and cost benefits associated with our latest FXT Edge filer demonstrate the massive advantages of a hybrid approach that can precisely match the storage media to the data being accessed,” said Ron Bianchini, President and CEO of Avere Systems. “And when deployed as part of our edge-core architecture, it also delivers the flexibility businesses need to locate storage where it makes most sense for the business.
Read the Full Story.
Over at Enterprise Storage Forum, Henry Newman writes that, while the industry has addressed storage complexity with NAS, SANs, and appliances, storage admins will require a whole new set of skills to meet the future challenges of application storage.
So if you are a skilled, highly talented administrator, what should be your plan to ensure that your salary does not take a nose dive? I think the answer is appliances for data analysis. (I am likely not talking about Hadoop, as many of the architectural designs for products in this area are completed.) Data analysis appliances are in their infancy today and will require significant care and feeding. The types of data analysis are going to be very complex. For example, you might de-pixelize an image and create a database of geolocations, normalizing for the resolution of the image, which might change over time based on improvements in technology. Then you might correlate the pixels to look for weather, climate or some other change like deforestation. This will be far different than taking business data and trying to correlate prices to sales to maximize profits.
Read the Full Story.
Our friends at Avere are offering a free copy of NAS Optimization for Dummies.
Big NAS performance comes from your ability to scale, eliminate sources of latency, and gain the advantages of the cloud. Get started with Avere Systems’ Special Edition of NAS Optimization for Dummies by Allen G. Taylor.
In this book, you’ll find:
- How to configure NAS storage for optimal performance
- Ways to reduce the cost of upgrades as your storage needs grow
- How to minimize the impact of multiple users hitting the storage systems at the same time
Read the Full Story.
In this video, Nebula CEO Chris Kemp discusses his new product called the Nebula One and the future of cloud computing with Cory Johnson on Bloomberg Television. Kemp was formerly the CTO of NASA IT.
Nebula One brings the cloud to you, under your control, behind your firewall. It is an integrated hardware and software appliance providing distributed compute, storage, and network services in a unified system.
The Nebula One has to be cool — they’ve got Patrick Stewart and Andy Bechtolsheim in their launch video!
Advertisement
insideHPC.com is a production of insideHPC, LLC. © 2006-2013 Sitemap
