DATA
What if we'd lost Magellan's log or the notes Darwin made on the HMS Beagle? Such records have always been painstakingly copied and then kept safely in libraries and museums. But in the digital age, organizations like NASA are so overloaded with data that the backup backlog is pushing the agency to a state of oblivion.
Every night, NASA's robots back up miles of magnetic tape containing the latest data from satellites and space missions. Every day, ongoing missions produce still more data, which is stored on additional tape. The new media hold so much data, according to Milt Halem, CIO at Goddard Space Flight Center (www.gsfc.nasa.gov) in Greenbelt, Maryland, that in a few years NASA will fall so far behind that it's unable to copy the tapes before they deteriorate. "It may take 20 years to read all that data," Halem says. "But the lifetime of the tapes is less than 20 years."
The problem is that, over the past decade, tape data-storage capacity has increased roughly in accordance with Moore's law. During the same period, however, data transfer speeds have increased at a rate of only about 1.3 times every 18 months. Currently, it's possible to read from 40-Gbyte tapes at a maximum speed of 12 Mbytes per second. "In two years, we'll be at 24 Mbytes," Halem says. "But by then we'll be using 100-Gbyte tapes. The time it takes to copy gets longer and longer."
Even if the agency immediately put newly made tapes in the queue to be copied, it may be impossible to complete the transfer before a nonrecoverable data loss occurs. Similar problems face other centers of scientific research, such as the European Laboratory for Particle Physics, aka CERN. Before long, the crisis will hit the next wave of large data-intensive organizations, from the Social Security Administration to banks and insurance companies.
Over the long term, Halem believes the government should encourage the development of next-generation mass-storage technologies, partnering public agencies with industry to maintain a research program.
In the meantime, NASA's only choice is to make do with existing storage technology, hoping it can rise to the challenge as the agency did during a similar crunch in the past. Weather studies from satellites launched in 1979 were placed onto tape that almost immediately became obsolete. It took two years and what Halem calls "a Herculean effort" to save them. They contained, among other gems, evidence of global warming and the first complete measurements of the 1983 El Niño, wisdom that otherwise would have been lost to the ages.
MUST READ
The eBallot and the Bullet
Distance Makes the Heart Grow Stronger
MP3 Meets MPAA
Ask Dr. Bob
Caught in the Act
Now That's Slugging Percentage!
People
Jargon Watch
NASA's Backup Backup
DNS Gets Tongue Lashing
Virtual Viral Marketing Virus
The Weather Overground
Raw Data
