By Seth Noble, PhD, CEO of Data Expedition, Inc.®
A colleague recently shared a link to this fascinating IEEE Spectrum article about the film industry's struggle with preserving digital media. We all know that physical film degrades over time, but digital is supposed to live forever. Or at least, stable media such as tapes are supposed to last for up to 50 years. Yet rapidly cycling standards have reduced the life expectancy of LTO tapes, the predominant format for digital archiving, to just 7 years. According to the article, some archives are actually printing digital movies to physical film stock, because it lasts up to 100 years and is less expensive to maintain.
Just hours after reading that article, I was on a call with a customer using our software to move several petabytes of archival media. Digging a little deeper into their workflow, I learned that they have been paying over a million dollars per year, per data center, just to maintain their existing LTO libraries. Not only was it expensive to keep LTO alive, the cost of pulling data out, certifying it, and securing its transport piled on even more costs and delays. And that still doesn't count the cost of adding new media.
The solution for this major media enterprise is to move everything out of their tape data warehouses and into a hard-drive array on another continent. It is a monumental undertaking that requires moving gigabits per second of data for months at a time. But the benefits, both short and long term, are well worth it. While individual hard drives can be short-lived and unreliable, well-managed storage arrays are reliable, predictable, and relatively inexpensive. Perhaps more importantly, they are easy to access and easy to replicate. Once everything is out of LTO, the media will be copied to multiple data centers on multiple continents. This provides levels of geographic redundancy and ease of access that LTO could not match, and at a fraction of the cost.
Some of the comments on the IEEE article called it alarmist. But when a top-tier media company abandons an entire infrastructure platform, it is time for everyone in the industry to take notice.
One thing is clear: digitizing media is no guarantee of longevity. The pace of technological change means that archived data cannot just sit on a shelf. To stay alive, data must be kept moving. That requires constant awareness of evolving standards and having the tools on hand to ensure rapid access and transport.
# # #
About the Author: Seth Noble, PhD, is the creator of the patented Multipurpose Transaction Protocol® (MTP™/IP) technology and a top data transport expert. He is Founder and CEO of Emmy Award-winning Data Expedition, Inc.