There have been lots of AI-based financial trading systems. This one from Hughes and Pareto is different. It works.
A lot of men will tell a woman it's her mind they're after. But in the case of Christine Downton and some men from the military-industrial complex, it was true. In her head was expertise that researchers at Hughes Electronics Corp. - missile makers, robot designers, spy satellite pioneers - wanted to tap. Enemy secrets? Weapons plans? No, the nitty-gritty of financial markets.
In 1993, Christine Downton, a star analyst at the British investment house Pareto Partners Ltd., flew out to Hughes Research Laboratories in Malibu, California, to upload her knowledge of the world's bond markets into a machine. That knowledge now sits in an Apple at Pareto's London offices, looking after funds worth US$200 million. Another clone of Christine will join it shortly, choosing the best markets in which to invest. Pareto and Hughes have decided that, in the war for the world's markets, the mechanized divisions are going to win.
Downton, a Pareto executive named Ron Liesching, and the rest of the Pareto-Hughes team believe that their artificial intelligence trading system - call it Robotrader - is one of the first concrete steps toward a shakeout of the financial industry precipitated by new technology. Computer-based AI systems will automate many of the jobs of analysts and dealers and destroy the closed shop in the upper echelons of finance. Wall Street fat cats will see their value plummet like share prices in a market crash; only those who embrace the technology will survive.
Plenty of Pareto's opposition, having followed AI's track record in the markets, will scoff at Robotrader. Scientists have long seen the markets as providing problems tailor-made for their technologies - complex, with multiple variables and large volumes of data that must be processed rapidly. Financiers have dreamed of magic tools with which to make their fortunes. As a result, lots of money and most of the contents of the AI tool box - expert systems, case-based reasoning, neural networks, and genetic algorithms - have been thrown at the problem. But the results have been disappointing. AI-based trading systems that start up in a blaze of publicity, like Citibank's neural network for foreign exchange trading, tend to have their plugs quietly pulled when they fail to live up to the press releases.
Liesching, Pareto's director of research, knew about the pitfalls: he'd suffered through some of them at County NatWest Investment Management, in London. He knew from the start that such projects take time and money - in this case, more than a year and more than $2 million. But he's not the sort of man to be deterred by that. He's as sweeping and startling in his predictions about technology's possibilities in financial markets as he is scathing about other people's failures to realize them.
In the early 1990s, Liesching started hunting for techie partners to help Pareto automate the stewardship of at least some of the $17 billion in funds it manages. Bell Labs, Digital Equipment Corporation, and Unisys were all found wanting. They had clever, powerful tools, but they didn't meet the peculiarly tough requirements of the financial world. "There's a high data rate," he says. "There's a lot of noise in the data, there are errors, it's not all numbers, and you've got to do the job reliably; if you're wrong you're gone."
Liesching's analysis sounds nasty, even hellish. Which is where the military comes in. War, after all, is also hell. "The military deals with dirty applied problems, just like you get in finance," he says.
He's not the first to spot the similarity. Sun Tzu's The Art of War does a brisk trade among business types - as does the US Marine Corps's Warfighting manual. In fact, last year the marines moved into the New York Mercantile Exchange, putting officers under training into the trading pits. You can see the similarities to the modern command post: lots of information but not necessarily enough, lots of decisions, and a lot riding on it all. According to General Richard Hearney, Marine Corps assistant commandant, they wanted to compare how the two professions dealt with the type of stress normally associated with the battlefield.
The similarities explain why both soldiers and financiers are eager to use AI. They worry about information overload; they also worry about emotional stress. Emotions, in Downton's view, are the trader's enemy. "Emotions distort people's rational judgments,'' she says. "There's a fear factor - people tend to make mistakes when they're losing money. They also make mistakes when they've made money, because they get bigheaded."
There are other human irrationalities, too, "cognitive biases," as Downton calls them. "The market will get fixated on one variable rather than a whole range." Individuals, she says, "get hung up on the most recent piece of information they received or some rather distorted assessment of the information - human beings just have processing limits."
Those limits are becoming ever more of a hindrance. Consider the recent research finding that people can process only about seven chunks of information at any one moment. Twenty years ago, when a financial analyst typically looked at only a few bits of data in three or four markets, this didn't matter; now it does. "If you want to compete, you probably have to cover about 10 to 15 markets," says Downton. "You might want to look at, say, 10 to 20 variables for each of three sources of return. You're looking at trillions of potential combinations."
Anyone who's tried to make sense of a Tom Clancy novel will know that the modern military is similarly complicated, which is one of the reasons that armies spend a lot on AI. Many of the key university AI labs were started - and are still funded by - the Pentagon's Defense Advanced Research Projects Agency, the incubator of the Internet. The image processing techniques used in machine vision, for example, have been used to analyze data from satellite cameras, radar, and infrared sensors. Missile developers have adapted tracking and pathfinding algorithms written for laboratory robots. Even the age-old chore of calculating troop movement logistics has benefited from problem-solving and expert systems programs.
The 1991 Gulf War most vividly displayed AI's usefulness. The "smart" bombs were not that smart - they mostly just homed in on splashes of laser light. But DART (Dynamic Analysis and Replanning Tool), a distributed-planning program developed at BBN Systems and Technologies, was very smart indeed. It proved invaluable in sorting out the scheduling nightmares of an operation as vast and sprawling as Desert Storm.
This is the background Hughes brought to the table. It also brought an eagerness to diversify from a shrinking defense market. The fit with Pareto seemed perfect and quickly developed into a real partnership. All that remained was to show it could really be done - that AI really could master the trader's art.
As a teacher of that art, Downton would be hard to better. She has studied the markets for 20 years as an academic and practitioner, including spells with the Bank of England, the US Federal Reserve Bank, and Liesching's old firm, County NatWest. That experience is coupled with a certain individual flair. Liesching vividly remembers their first meeting: among a bunch of Wall Street types in suits, Downton cut a striking figure with her bright red hair, jeans, and motorbike.
The man at Hughes assigned to squeeze out Downton's experience was Charles Dolan, who has a PhD in computer science from UCLA. Dolan likes to devote himself to what he calls "world-class-hard problems." At first, he wasn't sure that finance offered any; Downton convinced him. And the project did have broader appeal. As Dolan points out, "In the military, it takes 14 years to develop a new missile before it goes into production. By that time you don't see much of your technology, because it goes through so many transformations. In finance you see it right away." You also get to see your current technology - instead of the technology you thought was leading-edge some 20 years earlier - perform.
Dolan's approach to AI is a mixture of traditional symbolic logic and newer connectionist theories, in which intelligent behavior emerges out of an artificial "neural net." Dolan's view is that the two are part and parcel of each other - that within the brain's networks of neurons there is structure and that this structure is the embodiment of symbols. He tries to create such "knowledge spaces" on the computer, based on the symbolic structures that have been laboriously built into the wetware of his willing subjects.
To do this, Dolan developed a system Hughes calls M-KAT (Modular Knowledge Acquisition Toolkit) - software tools for extracting and encoding human expertise. M-KAT has been used to "knowledge engineer" military skills, such as how tank commanders plan an assault on an enemy position. By the time Downton arrived, Dolan and his Hughes team had become highly proficient at knowledge engineering. "It's hard to measure the efficiency of the acquisition of knowledge," says Dolan. "It's generally measured by how many 'chunks' of information can be extracted a day, where a chunk is defined as a fairly complex rule with four or five conditions. We were doing ten chunks a day on average - three to ten times the benchmark."
Because knowledge engineering means cross-examining the expert's thought processes, it often exposes charlatans. Downton proved to be the genuine article; indeed, "she had quite a bit more access to her internal thought processes than most experts do," Dolan says. Still, it took a grueling series of sessions spread over 18 months to get a fair sample of those processes, with Dolan switching from tool to tool to try to mimic the thought trains Downton described.
The most difficult part was capturing Downton's "feature extraction." "When I look at a variable,'' she says, "I ask questions such as, Is this inflation number high? Has it been high for long? and What are the recent trends? The most time-consuming part was explaining what I meant by 'high,' and then helping them design something that would look at a particular number and come up with the same assessment I would."
The result is a set of 2,000 rules called the Global Bond Allocation Strategy. From electronic market-data feeds, the system takes in around 800 items of economic information - things like countries' public-sector and current-account deficits, inflation rates, money-supply figures, and so on. After crunching through millions of permutations, it spits out conclusions as a series of recommendations, such as selling holdings in Denmark and buying bonds in Germany. The recommendations are passed to a flesh-and-blood Pareto trader, who then makes the deals.
Vilfredo Pareto was a 19th-century economist who pioneered the introduction of higher mathematics to economics. The company that sports his name is, fittingly enough, devoted to a "quantitative" approach to trading - financial jargon meaning that all its trading and investment is done using models, albeit simplified ones, of what is happening, rather than feelings and theories about why. As such, it seemed natural for Pareto to turn to AI - and AI fit into the firm easily. Robotrader produces recommendations like any of Pareto's other models, for which its traders must then find the best market price. It does so at a far more sophisticated level, to be sure, but it is fulfilling the same basic function.
So how has Robotrader performed? In the markets, the rate of return from trading is a function of risk: the more profit you want, the bigger the risk you must take. Pareto manages money for major public and corporate pension funds. Pension funds are generally conservative - they want low risks and will settle for lower returns. At the moment, Robotrader is managing mostly highly diversified portfolios with relatively low risk levels. On these, says Liesching, the system produces returns of around 3 percent above a bond-market benchmark - the kind of workmanlike performance that large pension funds seek.
The returns are not startling. But then, Robotrader isn't being asked to startle; the low risk levels are part of its (reprogrammable) parameters. And they are all the program's own work. Downton resists any temptation to override the system's recommendations, especially when the markets are volatile. That would defeat its whole purpose. "Few people are prepared to rely completely on analytical processes," she says. "They want to second-guess them in some way. That's when their emotions get involved. And it's probably just when they should be relying on their models that they are throwing them out the window."
This fits with her own recent experience. Though Downton and her silicon twin are nearly always in agreement, "sometimes there are slight nuances,'' she says, "between what it recommends and what I think I would do. But when I look into it, I see the machine is right in that it has noticed information I hadn't remembered, or it's more detached."
Alternatively, its success might simply be luck. No matter what techniques financiers use, there is always an element of chance in trading the markets - the dartboard that "picks" stocks better than the pros. Dolan acknowledges this, and suspects that many, if not most, of the success stories of using technologies to play the markets come down to luck: no one talks about the unlucky ones that fail.
But in the management of $15 billion, Liesching admits, there can be no reliance on serendipity. That's one reason Robotrader is managing mostly low-risk, highly diversified funds. A Pareto client, who directs a pension fund for one of the biggest US technology companies (which, like most of Pareto's customers, declines to be identified) agrees. "If you have $20 million and use the technology to pick 100 stocks to invest in and one screws up, it's only $200,000,'' he says. "But if the technology picks just five stocks for you to put the $20 million on and one screws up, it's $4 million gone. That's significant. If an investment manager screwed up like that I'd fire him the next day."
One company that thinks it has seen the future is Bermuda-based insurance giant Exel. It liked Robotrader so much that in April 1995, it bought a 30 percent stake in Pareto, with the intention of merging AI-based risk-management methods into insurance products. According to Exel vice president Gavin Arton, the company plans to try Hughes-Pareto knowledge engineering to automate some of its own underwriting expertise.
And Pareto is furthering its own commitment to AI "wherever appropriate," says Liesching. Shortly after the bond machine got up and running, Downton went back to Hughes for another bout of brain-draining, this time to extract her expertise in equities and their interrelationship with the bond markets. From this the Hughes-Pareto partnership has built a second knowledge-based system - its Global Asset Allocation Strategy. The system is currently undergoing final testing, with the firm trading its recommendations on paper to see how they would do. The next step is to go live with real money, and Pareto already has a customer with a $50 million portfolio signed up.
Others remain to be convinced of the success of the existing model, let alone the new one. And some wonder whether, even if Pareto has an edge, the ruthless efficiency of the markets might whittle it away. Another big Pareto pension-fund client points out that investment is not quite the same as solving scientific problems. "You are part of the problem,'' he says. "If your system picks profitable bonds, then the very fact that you purchase those securities affects the markets. And when you're managing $15 billion, your actions can move the markets. There's a feedback loop that causes your solution to become part of the problem."
Liesching is not too worried. He believes that AI - along with agent technology - will cut a swath through the industry, automating thousands of jobs or downgrading their skills, not necessarily because their results are that much better but simply because they're cheaper. "People in finance are generally overpaid and underqualified, and there's too many of them," he says. Most of what these people - analysts, strategists, marketing executives, and so on - do is what he calls "knowledge-directed searching." But because of the vastly increased data flows, that's becoming impossible.
For her part, Downton modestly says that no human could process the volume of information that the Global Bond Allocation machine sucks up. Indeed, Liesching believes that AI systems will lead to radical downsizing in the upper-middle ranks of the finance industry. One by one, the functions people perform and for which they charge huge margins will be picked off and automated: identifying arbitraging opportunities, building and optimizing portfolios, brokering, trading, and managing market risk. The Internet will hasten the process, delivering sophisticated services directly to the consumer.
Liesching's predictions seem to fly in the face of current trends, where human financial expertise has never been at a higher premium and Wall Street salaries climb relentlessly. But he is adamant that a shakeout is coming. "Whoever can replace these people with machines will win," he says. "Even if the machines are only half as good - they can work 24 hours a day, and they don't have the personality side effects.''
Downton has no worries that her clone will take her job. "It's enormously liberating," she says. "It releases the human expert from the drudge work of information processing." And it lets her spend more time thinking about the markets and less time immersed in them. "The best use for human insight is in designing models, not in second-guessing them."
It also gives her time to look for changes in the way the markets operate. As John Maynard Keynes remarked, when the facts change, it's time to change your mind - and Downton now has two minds to change, with a third on the way. As yet, she thinks, the only changes in the market have been superficial ones, with which the system's learning algorithms are perfectly capable of coping.
The machine may mimic an expert, but it isn't one; Christine Downton - capable of changing her mind - is.
That still gives her, and true experts like her, the edge. In the long run, the technology might capture the gift of developing expertise, or even cut away the need for it. After all, if all the traders are rational robots - not emotional, cognitively biased people with worries and fears and vanities - the markets might behave more efficiently, removing many of the cunning possibilities for arbitrage that experts can discover. Until that day, there's money to be made.
