Network-geek power couple Judy Estrin and Bill Carrico helped build the Internet as we know it. Now they want to safeguard its soul.
Lights burned at 3 am in the Stanford Engineering Research Lab through the winter of 1975, where a graduate student waited in an office crammed with PDP-11 minicomputers for a line printer to start clattering.
If the noisy machine kicked on, it meant that a trickle of data had found its way to University College London, and an answer had trickled back. Often the printer sat silent. The embryonic network - running a prototype implementation of TCP - crashed and crashed. Some nights, Vint Cerf, who was leading the Stanford project, dropped by to see how the testing was going. He felt confident the process was in skilled hands. Though the graveyard shift was considered grunt work by the other students, it was just the kind of hands-on, down-in-the-bits job that came naturally to Judy Estrin, who was the only woman in the lab.
Cerf had known Estrin since she was 12. Her father, Gerald - a mentor to many designers of the earliest computer networks, including Paul Baran - had been Cerf's own thesis adviser at UCLA. Technical brilliance ran on both sides of the family. Estrin's mother, Thelma, was a pioneer in biomedical computing, and one of the first women in the US to earn a doctorate in electrical engineering. Cerf still chokes up on the phone recalling how Gerald Estrin accepted him as a member of the family after Cerf's own father died.
Twenty-six years after those night shifts at Stanford, TCP - along with its sister protocol, IP - has turned out to be sufficiently robust to support more growth than its architects dreamed, like a scalable trellis upon which the rampant vines of the Internet flourished. Much of the reason the Internet has been able to scale so effectively is IP's decentralized, distributed architecture. Engineers like Baran, Cerf, and Robert Kahn mapped the Net like a modern non-hierarchical organization: Rather than imposing the top-down, circuit-switched controls employed in telephone networks, they put all the information required for packets to reach their destinations into the packets themselves, empowering routers to make their own decisions. This elegant simplicity bore fruit at the edges of the IP network, where grassroots developers were at liberty to rapidly evolve smarter devices and applications without impacting the core. The legacy of this approach is the creative profusion of new information services all around us.
The race for dominance among service providers, however, has brought market pressures to bear on the Net's deep infrastructure. In the world of TCP/IP, shipping packets is a cheap and unglamorous job - not an appealing business model for the swarms of carriers and vendors competing to capitalize on the boom in Net traffic. With so much bandwidth in place, vendors are clamoring to push such gold-plated services into the marketplace as quality-of-service guarantees, ensured bandwidth, and built-in billing. In addition, the next generation of applications, such as IP telephony and ubiquitous services, require more precise levels of traffic engineering than TCP/IP has historically been able to provide.
In the past few years, the pressure has intensified to thread top-down controls throughout the distributed network. Not surprisingly, many ISPs are eager to embrace circuit-switched architectures, modeled after the telephony world, to achieve new levels of traffic management - not surprising because more and more, these providers are the telcos themselves. Often, however, these top-down schemes require embedding greater complexity into the network core, such as more and bigger routing tables, and chunkier software in the routers. While providing short-term returns, many of these technologies turn out to be less scalable than they first appear, threatening the interoperability that enabled the Net to thrive in the first place.
A quarter-century after the seeds of the Internet came alive at 3 am in the Stanford engineering lab, Judy Estrin is convinced that these perennial infatuations with circuit-switched panaceas are symptomatic of a deeper disorder: a flagging commitment in both public and private sectors to evolving IP.
Now Estrin, along with her husband Bill Carrico, is headed down into the bits again, to defend the integrity of the global networks she helped create.
The saga of Estrin and Carrico is a high-geek love story unfolding at the center of the changes that have swept the IT industry. After two decades in the trenches of Silicon Valley, their combined résumé reads like a timeline of network evolution. They met in 1979 at Zilog, the seminal startup that shipped the first commercial router, developed one of the earliest LANs, and manufactured the Z-80, still one of the most widely deployed microprocessors in history. In 1981, the couple founded Bridge Communications, which pioneered technology for linking dissimilar networks. In 1988, they jump-started Network Computing Devices, the leading supplier of low-cost, graphics-intensive Unix workstations. In 1995, they launched Precept Software, ahead of the curve of multimedia on the Web.
When Cisco snatched up Precept in 1998, Estrin became CTO of the largest networking company in the world, with Carrico running Cisco's small- and medium-size business division - an experience they found both gratifying and supremely frustrating. At the summit of Cisco's growth arc, Estrin and Carrico bailed, telling reporters they were coming home to their startup roots. Insiders say, however, the couple was weary of working for a company interested only in shipping more and faster boxes, rather than tackling the emerging set of challenges to IP.
"They're not the kind of people who enjoyed trying to steer this huge battleship around," says Marnin Kligfeld, who's married to Estrin's sister Margo. "They're born to drive PT boats."
When the couple announced the launch of their latest venture last year - a company called Packet Design - the business model seemed, frankly, tough to wrap your mind around. The trade press referred to it as a networking company, but would it sell networks? No, they explained, when they wanted to bring a product line to market, Packet Design would spin out another company to do that. So was it an incubator? "I don't believe in incubators," Estrin declared to a reporter last April, "although there is a place for incubation as part of something else."
Perhaps the best way to understand Packet Design is that it's built like a PT boat: small, agile, quiet, rigged for reconnaissance and deadly accuracy, packing more geek firepower than anything else on the water.
For the last year, Carrico and Estrin - or Bill and Judy, as they're known throughout the Valley - have been skimming IP expertise from companies like Cisco and academic institutions like the Information Sciences Institute, packing the Packet Design roster with such network gurus as Van Jacobson, whose flow-control algorithms successfully averted catastrophic traffic jams on the Net in the late '80s. (Creator of the widely used IP operations tools traceroute, pathchar, and tcpdump, Jacobson may be immortalized somewhere on your hard drive - most PPP software offers Van Jacobson header compression as a speed-boosting option.) Kathleen Nichols, formerly the director of Cisco's Advanced Internet Architectures division, also climbed aboard. She now leads Packet Design's research team with Jacobson (who is her husband), while co-chairing the Internet Engineering Task Force's working group on differentiated services. Cengiz Alaettinoglu, who co-chaired the IETF's Routing Policy System group, signed on last year, as did Steve Casner, one of the primary architects of the Mbone.
The first Packet Design spin-off, Vernier Networks, offered a glimpse of the company's technical strategy. Existing technology for deploying corporate-scale wireless LANs was weak, particularly in the areas of administration and security. Vernier products fix that by tapping the network at the routing level, offering IT managers control and security on wireless LANs comparable to landline networks. They allow transparent roaming across subnets within a company (using laptops, PDAs, PCs, pagers, or phones) without having to install client software in each device. The first Vernier boxes shipped this fall.
Vernier served as the initial proof-of-concept for Packet Design's spin-off model, making it clear that the company would not be sitting on its hands while the network gods cogitated in the back room. The next project on Packet Design's rollout schedule, to be announced at year's end, will cut even closer to Estrin and Carrico's core mission.
Called Packet Design CNS, the couple's new venture will focus on building tools and technology to enable large-scale service providers and telcos - think Sprint, Verizon, Qwest, and WorldCom - to boost IP traffic analysis, traffic engineering, and traffic management to higher levels without relying on quick fixes modeled after the telephone networks. CNS will also tackle security issues, providing new lines of defense against denial-of-service attacks and equipment failure. CNS won't be in the hardware business, though it may offer off-the-shelf Unix boxes with its software preinstalled. Unlike Vernier, the new venture will not be spun out of Packet Design, but instead run as a business unit within the mothership, close to Jacobson and Nichols' research team. Packet Design says CNS products will come to market within a year.
Estrin and Carrico compare the current state of the Net to the covered-wagon era of transportation. CNS, they say, will be a significant step toward putting cars on the road with power steering and automatic transmission.
Dave Passmore, research director of the Burton Group, calls Packet Design "the keepers of the Nethead religion. They're leading the charge."
History seems to be on their side. In the early '90s, the telcos trumpeted a circuit-switched technology called ATM as an end-to-end solution that would inevitably supplant IP-to-the-desktop. ATM is now regarded as a tool of limited usefulness in core networks, unsuited for the bursty and unpredictable traffic patterns on the Internet at large. Recently, vendors have advocated another virtual-circuit scheme, multiprotocol label switching, as a cure-all, but serious security problems have emerged with MPLS.
It's becoming clear that, as data and telephony networks converge, packet switching will provide the most effective and scalable foundation for the next generation of services. The danger, as Estrin puts it, "is that we will end up with the worst aspects of both worlds."
Various factors have contributed to the lack of forward momentum in the evolution of IP. Commercial research into improved routing algorithms fell by the wayside as switching technologies like Ethernet took off. Academic research in the past decade has favored let's-build-a-new-network-from-scratch approaches, in keeping with the belief that the Internet is already "done." In particular, there has been little evolution of what's called the routing control plane, which includes the ways that routers acquire knowledge to make decisions given current conditions in the network. Many of the IP refinements Packet Design is planning to introduce via its various spincarnations will be in the control plane.
Telstra's Geoff Huston, author of the ISP Survival Guide and co-chair of the Internet Engineering and Planning Group, concurs with Packet Design's assessment of the stalled state of routing research. "There has been very little development of routing protocols for some 10 years or more. Considering the changes in volumes, speed, and diversity of traffic that occurred in this interval, it's remarkable we've been able to scale these protocols as far as we have," he observes. "In the past, this work was within the realm of public funding channels, and the problem is that to get your proposal funded, you have to make it look like some wonderful new initiative so that it stands out from the crowd."
Part of the reason Estrin and Carrico are relieved to be out of Cisco is that they no longer have to be trophy geeks in a company that, to serve the short-term needs of its clients, had to take an agnostic stance on IP. In January's Cook Report on Internet, Estrin told analyst Gordon Cook: "Cisco will very publicly say a key part of their culture is that they have no technology religion. What they mean by this is they are not going to be so religious about IP that they won't ship ATM if the customer would really rather have it. What often happens though is 'no technology religion' can get mapped into 'no passion for a specific technology solution.' There are certain hard problems that, unless you have the passion to keep pursuing a solution, you will not overcome the obstacles that need to be overcome. When you hit a wall, you simply go off in another direction."
When I asked Kathleen Nichols what her colleagues had in common beyond their commitment to IP, she replied, "We're all flaming pragmatists. We're attracted to what works. That's where we see the beauty."
Sitting one afternoon last June in the unapologetically bland offices of Packet Design - where the only trace of corporate schwag is geek rigueur baseball caps - Estrin drew a line across a piece of paper. At one end of the line, she wrote A, for academic institutions doing blue-sky network research. At the other, she wrote B, for product companies, which she described as being in the business of "solving yesterday's problems." Then she made a mark in the middle. "We want to be here."
Packet Design's strategy for tackling tomorrow's problems is to first take a deep look into the marrow of today's networks. For the past year, they've quietly signed partnerships with ISPs and router vendors like Cisco and Juniper to run data analysis on the traffic moving across their backbones and core networks. Call it field research down in the bits. What they found, in several cases, was surprising.
Take jitter. When packets arrive so much out of sequence that they cause inconsistent delay at the receiving end, it's called jitter. This delay doesn't matter if you're browsing the Web, but it's poison to IP telephony and broadband multimedia. The prevailing understanding among service providers of why jitter happens is congestion: hot spots in the network where traffic surges max out available bandwidth. What Jacobson and Nichols' team found, however, is that the primary cause of jitter is not congestion, but configuration and implementation bugs. That's the kind of information an ISP fixing to spend an extra $10 million on boosting bandwidth might be curious to learn. Estrin calls this close analysis "myth busting."
One of the major carriers that invited Packet Design to x-ray its backbones is Qwest. Shankar Rao, manager of Qwest's IP architecture and development group, says he was particularly interested in employing Packet Design's myth-busting expertise to examine his company's assumptions about network resiliency, and the viability of pure IP strategies for guaranteeing quality of service. Rao contacted Packet Design after being impressed by one of its presentations at a meeting of Nanog, the North American Network Operators' Group, one of the geek-to-geek forums where the company is making some fruits of its research available for free.
"What Packet Design has is talent," Rao says. "The work they're doing is original - nobody else is taking this tack." He notes that the level of analysis available from Packet Design is hard to come by anywhere else. "Unfortunately, we don't have the resources to do this kind of empirical research. All the discussions we've seen about technologies like MPLS have been vendor-driven. Vendors push their own technology. They're not going to tell you 'You don't need this.' We believe in preserving the simplicity of the core network, and that the challenges we're facing can be solved with IP tools. Packet Design's vision is something Qwest believes in."
To comprehend that vision, you have to understand what's behind the name of Packet Design's latest venture. When I first heard CNS, I assumed the initialism stood for "central nervous system," a plausible moniker for a company aiming to diagnose ills in the distributed brain of the Net. In fact, the name comes from a metaphor Jacobson coined several years ago, in one of those moments when a technically apt image rises toward poetry.
On IP networks, packets move in clouds, rather than along easily predictable pathways. On circuit-switched architectures, however, end-to-end connections must be held open for the duration of each data transmission, which Jacobson dubbed strings.
CNS: clouds not strings. Clouds not strings, you might say, is Packet Design's rallying cry to the IP troops.
Nichols explains, "Even though people have a mind-set that it's simpler if you know where the packet is going, when you think of the complexity of a worldwide Internet, specifying the path is more complex than having a network that robustly handles the packets."
Estrin draws a cloud on a piece of paper, with streams of packets flowing in. "Circuit guys are used to tracing the path and looking at it," she says. "When you can trace the path, you're not using bandwidth efficiently."
Carrico added, "I frankly think it makes people queasy that they don't know where the packet is."
By bundling Jacobson-grade expertise, innovative products, and pervasive influence in the IETF and other groups, Packet Design is positioning itself as a top-to-bottom powerhouse to steer the Net through its current phase of development by boosting IP functionality to the next level.
Given current market conditions, the company's unusual two-layer structure seems like adaptive evolution: The spin-offs deal with the pressures of getting products out the door, while the research group is freed to work on an open-ended timetable. This hybrid model also allows Packet Design to take a grander role in the Internet community: With the lab stacked with IETF members, Packet Design's IP church will be on cozy terms with the state of standards development.
Inasmuch as the company leverages Estrin and Carrico's reputation as serial entrepreneurs ("they could launch a catering company and get funding," one associate remarks), Packet Design's business model can also be seen as a response to the fates of the ventures the couple has launched in the past. If success is measured only in acquisition dollars, all of their businesses have been successful. In retrospect, however, all that black ink is footnoted by missed opportunities. In a moment that must have been acutely bittersweet, Cisco cofounder Sandy Lerner once thanked Estrin for selling Bridge to 3Com, where Bridge was functionally buried, leaving a breach that allowed Cisco to become Cisco. Precept's products were too early out of the gate to really make their mark - an error Estrin once attributed to reading white papers, rather than staying close to the state of deployment of streaming technologies.
Packet Design's myth-busting activities will keep its startups grounded in real-world network operations. And while Bill and Judy will make spin-offs available for acquisition, the core company, they insist, will never be for sale.
Their employees commonly refer to the two-headed animal sitting in the executive chair as the Bill-and-Judy show. A longtime business associate observes that while Packet Design is "the perfect business model for Bill and Judy, I doubt it could be ported anywhere else."
To see the Bill-and-Judy show in full effect, I sat in on a Vernier planning session two floors down from Packet Design. Though Vernier CEO Doug Klein sat at one end of the table, the center of gravity was clearly on Estrin and Carrico's side, as all chairs shifted slightly in their direction.
The meeting was genial, an exchange of information between peers, with easy flurries of laughter that didn't seem for my benefit. "There's a level of bullshit in other companies that's missing here," one employee told me. "People don't have to do little things to constantly prove their value."
Friends who've known Estrin for a long time say that this atmosphere of mutual respect is reminiscent of the environment she grew up in. "Judy was shown that you don't have to stomp all over people to make progress," Vint Cerf recalls. "Thelma is quite blunt - diplomacy doesn't hold a high value for her. Gerry was never afraid to confront ideas, and tease out what was broken. That pragmatic notion of dealing with the real world pervaded the Estrins." (Margo Estrin became a medical doctor, and younger sister Deborah a computer science professor at UCLA.)
At 46, Estrin looks 10 years younger. During the course of the meeting, she paid attention with her eyes, turning her body toward the person speaking. When the name of any employee came up, she had at her command detailed knowledge of that person's particular talents. By contrast, Carrico kept his arms crossed, listening through the top of his head, cocking an ear when someone spoke to him, but looking up only when he, himself, was speaking. A question about a piece of documentation arose, and Estrin suggested expanding it, adding more information than necessary about a product's interaction with the customer's networks. "From a comfort level, that's what they need," she explained.
With her firm manner of focusing discussion and recall of personal details, Estrin would have made an excellent high school science teacher - the "strict but fair" one. Carrico, at 51, has something of the air of a highly effective principal.
In the old days, the Bill-and-Judy show had the flavor of a good-cop/bad-cop routine, with Carrico acting as the show-me-the-results businessman and Estrin the approachable people person who always remembered to return a favor. The joke was that it was the Punch-and-Judy show.
Jack Bradley, Packet Design's CFO - a robust, barrel-chested guy who has worked for the couple since Bridge - recalls one meeting on the eve of a preliminary filing to the SEC: Carrico told an accounting partner that, if the way he ran the numbers required a restatement of results, "Bill would have him hung from a tree and watch the crows peck his eyeballs out." NuvoMedia founder Martin Eberhard and CEO Doug Klein vividly recall "the board meeting from hell" in 1999, when Carrico was ambushed with a scheme for rejiggering NuvoMedia from a product company into a content portal.
"Bill was pounding on the table, 'It's not a real business!'" says Klein, adding, "The only time Bill and Judy were ever truly out of style was during the dotcom thing."
Eberhard learned to depend on "shouting matches" with Carrico to keep the business on course. "He was stubborn in the moment," Eberhard recalls, "but the next time I'd see him, he would have thought things through, taking into account every bit of data he could get his mitts on."
Former employees say that the birth of Bill and Judy's son, David, in 1990 has had a mellowing effect on Bill - though it wears off quickly when Judy is out of town. The couple has always been very guarded about the details of their personal lives, but perhaps no more so than any of the Valley's other multimillionaires. (One receptionist at Bridge wasn't even aware of their relationship.) They could have retired comfortably after 3Com acquired Bridge for $235 million, but as Estrin explains, "We don't retire well."
They have few close friends outside of the industry, and their personal indulgences have been spare: a second house in Aspen, a birthday Ferrari for Carrico, sold shortly thereafter. Unlike most of the Valley's wealthy, they have never fenced themselves in behind a wall of servants, A-list parties, and out-of-the-way tables at Il Fornaio. Even before Estrin hired him at Packet Design, says engineering manager Mark Tarpenning, he had run into her dozens of times at all the local geek hangouts: the Coffee Society, the Printers Inc. Café, Hobee's, and Fry's Electronics.
"I see Bill every time I go to Fry's. Here are two of the most successful and influential people in the Valley, and they're at Hobee's and Fry's," he says, adding, "I've never run into Larry Ellison there."
At home, Carrico relaxes by listening to Bach, Steeleye Span, and Bob Dylan on audio equipment that he builds himself, preferring the warmer sound of vacuum tubes to digital components. This hobbyist's enthusiasm hints at a deeper truth about his relationship with Estrin: Although Carrico is often described by the press as the businessman in the relationship, the fact is, they're both geeks. Manny Fernandez, who ran market research firm Gartner for the last decade, knew Carrico before he met Estrin, when they both worked at Fairchild Semiconductor in the mid-'70s. Fernandez encouraged Carrico to leave Fairchild and go to Zilog, which incubated an entire generation of Valley networking talent. Estrin joined the startup straight out of Stanford, turning down offers at Hewlett-Packard and Xerox PARC because a friend told her "The smartest people I know work at Zilog."
Estrin was managing a project code-named Ariel, a pioneering effort to build an Ethernet-like LAN. Carrico, who had signed on to the corporate-strategy team, strolled over to the systems building to see what the geeks were up to. Estrin admits that, at the time, "I thought marketing was an unnecessary evil." But Bill clearly got Ariel, grasping how important it could be for Zilog's future. They talked for hours.
And that was how one of the most durable romances in the Valley began - with a debate over the fine points of LAN infrastructure. "Some people are more fluff than substance," Estrin recalls. "Bill was substance."
As they've matured together over the years, they've gradually exchanged roles and attributes. Estrin has became a skilled marketer, having learned that "even if you have the technology, if you're not aligned from a business perspective, you're irrelevant." Cerf's former graduate student now sits on the boards of FedEx, Disney, and Sun.
When I asked Nichols the obligatory question about whether she regarded Estrin as a positive role model for women in IT, she shot back: "Judy's just herself, and she's highly successful. She's a positive role model for men."
A 1993 profile of Network Computing Devices in Upside declared that Estrin and Carrico "both seem comfortable with their working relationship. He is the boss, and she is second in command." A month or so later, Estrin took over as CEO - the post she held at Precept and now occupies at Packet Design. And Carrico, as chair, works closely with the engineering and research teams day to day, clearly at home down in the bits.
Watching them side by side is slightly uncanny. After being together nearly 24 hours a day for 21 years, they interrupt each other all the time, but it sounds like two seasoned jazz musicians, extending each other's phrases in tandem. You begin to feel that the same consciousness is looking out at you through two pairs of eyes: a single duplex mind - the subtlest network of all.
