Sent: Tues, September 11, 2001, 8:59 a.m.
Subject: 1 WTC -- plane into tower
Are you guys OK? Someone let me know.... Thanks....
Matthew Farley, partner of a law firm on the World Trade Center's 89th floor, was on a train into New York City when the first jet hit Tower 1 on September 11, 2001. Farley sent and received this e-mail and 90 others on his BlackBerry PDA as he tried to account for his staff.
Farley's story is just one example of the thousands of e-mails, text messages, weblogs, videos and other digital snapshots captured in The September 11 Digital Archive.
"Sept. 11 was the first major event of the Internet age," said Tom Scheinfeldt, managing director of the archive. "For the first time in history, people experienced this major world event online, through all kinds of media and computer technologies. Someone had to quickly mobilize to preserve a digital record of the attacks."
While previous world-scale events such as Pearl Harbor were primarily documented by vast written records, the majority of materials documenting public reaction to Sept. 11 are now online.
"An amazingly important part of the story is how the Web changed itself," said Lee Rainie, director of the Pew Internet & American Life Project.
Researchers from George Mason University and the City University of New York have amassed more than 20,000 pager messages, e-mails, digital images, websites and other digital materials since the project launched last January (2,000 of which are currently available on the site).
Researchers hope to use the archive to develop free software tools to help historians to do a better job of collecting, preserving and writing history.
However, security concerns have made it difficult for researchers to gather personal stories and digital materials from individuals directly involved in the Pentagon attacks. That material is still classified as an FBI crime scene.
"There was a certain amount of secrecy involved," Scheinfeldt said. "But by and large, we've been successful (in gathering materials) even in light of security concerns."
Another project, The September 11 Web Archive began spidering the Web just hours after the attacks to cache corporate, government, media, memorial and tribute websites.
The archive, a collaboration between the Library of Congress, the Internet Archive and webArchivist.org, finished culling the Web last December.
"They got a fairly broad, deep snapshot of the Web in those couple of months (after the attacks) ... before the government security apparatus was up and running," Scheinfeldt said.
"We wanted sites that were international in scope and content, showing a variety of points of view," said Diane Kresh, director of public service collections for the Library of Congress. "We don't censor what we collect. This is not just collecting an American point of view."
Since the attacks, government and security agencies have removed sensitive information from the Web, such as National Pipeline Mapping System data, chemical site security reports and other materials.
"We're nearing one year since the attacks and we have yet to see any coherent policy from the government on what information should come down and what information should go back up," said Gary Bass, executive director of OMB Watch.
The Library of Congress hasn't deleted any websites from the September 11 Web Archive. However, individual site owners can block public access to sites in the archive using a robot exclusion file.
For example, the Nuclear Regulatory Commission Web page is no longer accessible in The September 11 Web Archive because the agency has blocked public access to the page.
While the extent of archived materials that have been blocked from public access is unknown, critics say that alterations could stymie researchers from capturing a complete digital record.
"The archived sites stand for what our open society demands," Bass said. "It's not just history, but it's also creating a link from history to the current time. If you distort that, you distort democracy."
Even if government agencies block public access to sensitive data from the archive, chances are that it can be found on search engines like Google and sites like OMB Watch, which have stored cached versions of government websites.
"The Web is not like a faucet that you can turn on and off," Bass said. "If anything, it's like a leaky faucet."
"You cannot cache the whole Web," Rainie admits. "This archive is big and ambitious and it doesn't remotely cover everything that's happened online that day on every website."
Still, archivists say that the enormous amount of primary source material available in the archive far eclipses the number of sites that have blocked public access.
"They are, in essence, a snapshot of a particular point in time, in history and the development of the Web," Kresh said. "I don't think that the limits (of the archive) have yet been determined."
Researchers will relaunch the archive this week. A sample of about 2,000 websites will be searchable by categories of site producers and the types of user action enabled by sites. New reports on the use of the Web after September 11 will be available in the Analysis section.
The revamped collection will help people make sense of the archive and how people used the Web after Sept. 11, said Steve Schneider, one of the researchers for the project.
"The production community on the Web, in times of crisis, can make the Web useful in ways that we've never seen before," Schneider said. "I suspect we'll see it again in the future."
"Every time there will be a major news event there will be some effort to gather primary resources as they are happening," Rainie agreed. "We hope (the Web archive) will be of significance many generations hence."
