Big Data over Big Distance: Zettar Moves a Petabyte over 5000 Miles in 29 Hours

Print Friendly, PDF & Email

Today AIC announced a world-record in data transfer: one petabyte in 29 hours encrypted data transfer, with data integrity checksum unconditionally enabled, over a distance of 5000 miles. The average transfer rate is 75Gbps, or 94% utilization of the available bandwidth of 80Gbps.

Zettar has been capable of handling multiple PBs of high speed data transfers weekly for several years” said Dr. Chin Fang, Founder & CEO of Zettar Inc. “Even with massive amounts of data, this test confirmed once more that it’s completely feasible to carry out long distance, fully encrypted and checksum-ed data transfer at nearly the line-rate, over a shared and production network. As is, there is a physical limit of 80Gbps bandwidth cap in place. Also, the transfer every now and then must deal with other contending traffic. Otherwise, the rate would have been even higher.

A wide range of industries will need to take advantage of Exascale computing, oil and gas, media and entertainment, life sciences, defense and intelligence agencies, large retail businesses, to name just a few. Newly emerging fields such as artificial intelligence, machine learning, smart cities, and autonomous vehicles are no exceptions.

The test sent data along a unique 5,000-mile loop that goes from the SLAC in Silicon Valley to El Paso to Houston to Nashville to Atlanta, and then back to SLAC in California. This is the world’s only 5,000-mile 100Gbps loop available and has been established by the Energy Science Network (ESNet) for the SLAC/Zettar’s efforts since 2015. ESNet is operated by the DOE’s SC. It runs the world’s fastest network for science and connects DOE SC laboratories together and to the other parts of the Internet.

Transparency is critical when it comes to evaluating high-speed data transfer milestones like the recent successful 1-petabyte effort. The test performed by Zettar is likely the only one in the world whose results were publicly viewable (and can be seen on the ESNet’s Network portal for a limited time).

AIC will host an exhibit at SC18 in Dallas. Visit booth #4153 to learn more.

Check out our insideHPC Events Calendar