Sign up for our newsletter and get the latest HPC news and analysis.
Send me information from insideHPC:


DDN and IBM Team on 50 TB per Day, Inter-Continental Active Archive Solution

ddn2Today DDN announced that Yahoo Japan has deployed an active archive system jointly developed by DDN and IBM Japan. The new system allows Yahoo! JAPAN to cache dozens of petabytes of data from its OpenStack Swift storage solution in a Japan-based data center, and transfer data to a U.S.-based data center at an astonishing rate of 50 TB of data per day – thus enabling energy cost savings of 74 percent due to lower energy rates in the United States versus Japan, while ensuring fast data access regardless of location.

At Yahoo! JAPAN, we were grappling with a number of challenges related to our large, fast-paced data growth and vital disaster recovery needs; however, installing a massive storage system in a data center in Japan raises additional issues such as power consumption,” said Mr. Daisuke Masaki, Cloud Innovations, Site Operations Division, Systems Management Group, Yahoo Japan Corporation. “We therefore opted for a bold technical solution in which our data center in Japan caches data from the existing object storage (OpenStack Swift) and saves the data to a data center archive in the United States, which can be operated with 26 percent of the electricity cost of a data center in Japan and at about one-third of the cost of competitive solutions. Moving forward, we plan to expand and save data from multiple websites in Japan to the active archive system in the United States.”

Yahoo! JAPAN introduced the active archive system, which allows data to be cached in a data center in Japan and then transferred and stored in a U.S.-based data center owned by YJ America, Inc., its local subsidiary in the United States. The decision to move to an active archive system configured across dual continents was based, in large part, on Yahoo! JAPAN’s rising data volumes, multi-petabyte data backup requirements, and the disaster recovery measures it implemented after the Great East Japan Earthquake.

The joint DDN/IBM Japan system is able to handle dozens of petabytes of data within a single file system configuration. It allows the data center to cache data from an operating object storage (OpenStack Swift) private cloud at a fast rate of 11TB/ hour and back up the data in a data center archive in the United States at a breakthrough transfer rate of 50TB per day. The system allows users in Japan to access data and conduct services easily and without concern for where the data resides.

The active archive is built on DDN’s high performance SFA7700X® storage solution and IBM Spectrum Scale, which together provide the high-speed input/output (I/O) performance required by Yahoo! JAPAN. The data is cached on a solid state drive (SSD) that enables higher-speed I/O and ensures expedited metadata handling. The solution enables data sharing by using the caching function via the IBM Spectrum Scale Active File Management (AFM). In addition, the high density DDN SFA7700X ensures ample capacity and scalability to meet Yahoo! JAPAN’s fast-growing, high-volume needs.

Sign up for our insideHPC Newsletter

 

Resource Links: