Participants in the North American Network Operators Group email discussion list -- the primary shop talk forum for telecom and ISP engineers -- have been bandying about ideas for a new T-shirt.
One prototype features the black profile of a backhoe, the staple machine of construction crews everywhere. A massive shovel hangs off an insect-like arm, suspended above four giant tires. It's not the image you'd expect for folks who deal in circuits, switches, and packet flow. But there's something else: The backhoe is positioned within the crosshairs of a firing device.
Do we have a problem with backhoes here?
"Basically, what backhoes do is they hit some [fiber-optic cable], they pull the shovel back up and bring the cable back up with it -- and snap the cable," said network engineer Sean Donelan. "If you're lucky, it's a clean snap... If you're not so lucky the cable actually stretches, and it may break in several places."
Lest it be forgotten that miles of copper and fiber-optic cables underpin our communications infrastructure, the backhoe and its toothy shovel serves to remind us. The fiber pathways underfoot are where the sexy ether of communications comes down to muddy earth.
Donelan is a network engineer for a nationwide internet service provider, Data Research Associates, that serves state governments, libraries and schools. He's seen fiber cuts all over the country. In fact, the ISP has had more trouble from fiber cuts than from hurricanes.
The machines are one of the biggest "facility" problems for network engineers right now, Donelan says. Network engineers, he explains, are the ones "sitting in the network operations center watching their maps turn red" when the backhoe strikes.
Hence, the antibackhoe T-shirt.
Their anger isn't misplaced, either. Ron Olitsky is president of the Southern California chapter of a 67-member, nationwide network of "One Call" centers aimed at preventing underground damage of all kinds by excavators.
He confirms: "The vast majority of dig-ins do occur with backhoes."
The big shovels can be doing any kind of work when they rip into glass. Interruptions have been caused by road and railroad construction work, water and sewer pipeline installations and repairs, guardrail installations, and clean-ups after train derailments. Then there are the farmers doing drainage work, landscapers installing sprinkler systems on golf courses, and individual property owners planting trees.
Such is the snap, crackle, and pop of communications' Achilles' heel.
Ron Rosencrans collects newspaper clippings and direct reports of dig-in infrastructure damage of all kinds as editor and publisher of Underground Focus. His list of outages, though growing out of date, is as long as it is repetitive:
A telecom cable cut severs phone service for 22,000 businesses and residences for up to a week; a cable serving a sheriff's dispatching center for emergency calls is cut; Internet service is interrupted in Los Angeles; 50,000 long distance customers lose service after a railroad construction crew digs through a 30-fiber cable; nearly 1.5 million lose service when a contractor cuts a long distance backbone cable carrying calls for Sprint, MCI and three other companies.
The shovels don't always have to have an engine at the other end, either. Rosencrans reports that Sprint's worst outage of 1997 was caused by a golf course worker repairing a sink hole, who drove a spade through a thumb-size, 38-strand fiber optic cable. It blocked 366,000 calls, with people in several states served by a Kansas City hub losing long distance service for as long as eight hours.
"Is it an issue? You bet it is. And it's a growing issue," Rosencrans said.
The reason it's a growing issue may not actually be due to a growing number of backhoe-meets-fiber events, however, but because of the ever-expanding -- and concentrated -- fiber infrastructure.
"Pipeline and telcos have always been familiar with [the problem], but now everybody and his brother has fiber," says telecommunications consultant David Lesher. Among them: network service providers, cable companies, companies competing with local phone companies, and traditional long distance carriers.
"There are a lot of projects going on out there," adds Olitsky.
Though the number of breaks may even be holding steady, the potential impact of a single break is always going up. "A few critical routes, when they get hit, can disrupt a whole lot more people," said Donelan of Data Research.
Western regional telco US West experienced four outages over the course of a year due to fiber cuts. But Mike Daly, manager for cable damage prevention for the fourteen-state carrier, agrees that it's not the number that counts.
"Fiber is the highest order of business because it's the most capacity you can lose in a single event. A 44-fiber cable could carry most of the telephone traffic going in and out of San Francisco on one line. And when a cable of that capacity or greater gets cut, the effect can be traumatic.
"Phone service, Internet, credit card transactions, cash machines -- it just goes on and on the things that you can't do if you lose a major route." Whether the effect is major or minor in an incident is pretty much up to chance, he says.
Interruptions from fiber cuts are "more noticeable because more people are using it," adds Donelan. "But I don't think the actual problem [of backhoe digs] is getting worse."
As the underground infrastructure gets more dense, there is a major effort underway, led by the federal government to cut down underground digging damage to service. It's meant to address service interruptions of all kinds, gas and petroleum lines being the most dangerous.
Next week representatives from the US Department of Transportation, construction contractors, telecom carriers, pipeline companies, railroads, and others are getting together to discuss implementation of the Comprehensive One-Call Notification Act, legislation signed into law 9 June 1998.
One-call centers let a contractor about to break ground make "one call" describing the location of the dig. The center uses a database to notify utilities with underground facilities in the area. The utility goes out to mark the underground location of their cables, pipes, etc.
One-call centers are meant to reverse the most common cause of dig-in interruptions. Ironically, it's a lack of communication.
"Studies show that in forty percent of the damage cases the contractors didn't call the one-call center," said Rosencrans of Underground Focus.
The One Call Act provides a comprehensive round-up of "one-call" practices that can help prevent accidents. It also provides incentives in the form of federal grants to states and local one-call systems to develop efficient one-call programs.
Donelan says the focus on human error is correct -- in his own experience, fiber cuts have been the result of contractors either not calling or calling but then digging in the wrong spot, he said.
"It's just lots of operator error. I don't think we'll ever get the human out of the system, even if we wanted to."
