Locating Osama bin Laden took years of painstaking intelligence collection and analysis. And despite what you see on 24, there is no magic. Each piece of the puzzle had to be fitted together over time, something that doesn't happen easily or automatically — especially when the intelligence is used to back a decision send people in on the ground instead of dropping a bomb or firing a cruise missile.
But the seemingly insatiable hunger for magic has popped up in the renewed slew of press coverage of a 2009 paper, "Finding Osama bin Laden: An Application of Biogeographic Theories and Satellite Imagery," by UCLA geography professors Thomas Gillespie and John Agnew. Typical headlines describing this paper include "Geographers Had Predicted Osama's Possible Whereabouts" and "This Class of Geography Students Found Bin Laden's Hideout Long Before the CIA."
These stories are simply wrong. By failing to describe what the paper actually says they both distort the real work that went into finding Osama bin Laden and cloud the question of how the techniques Gillespie and Agnew describe can be used in the future.
It's important to point out, as Danger Room did when the paper came out in February 2009, that the paper was an academic thought problem, and was not an actual mission to find Osama bin Laden. If the UCLA team was using their methodology to actually try and find bin Laden, they would have probably incorporated many of the criticisms I will outline, and may have gotten better results.
So, did the paper say that Osama bin Laden could be found in a compound built in Abbottābad, Pakistan?
No.
In fact, the paper said that the most likely location for bin Laden's hideout would be in Parachinar, a city about 268 kilometers away from Abbottābad.
Second, if the paper didn’t say that bin Laden would be in Abbottābad, did the paper at least say that bin Laden was 88.9% likely to be there?
No.
In fact, the paper said that the likelihood that he would be somewhere within 300 kilometers of Tora Bora would be 88.9%.
Actually this was never spelled out, but you can derive it the from the formulas in the paper, where P(d) = 0.99959058977238*d*, d equals distance in kilometers, and P(d) is the probability that bin Laden could be within distance d from Tora Bora (Math is fun! Also, when I do the math I get 88.44%, but who’s counting?).
If this seems like common sense, it should. One of the things that the distance decay model that the team used basically says is that when people (or animals, or plants) are on the move, you are more likely to find them closer to where you last saw them than farther away.
But even though Abbottābad falls within this area of 88.44% probability, so does Islamabad, Kabul, and numerous other urban areas. And the model doesn’t give any indication of the likelihood that bin Laden in one of those places or any other point in the 282,743 or so square kilometers defined by the 300 kilometer radius circle whose center is at Tora Bora, only that he is 88.44% likely to be somewhere in that circle.
So Gillespie and Agnew did not claim that they knew where bin Laden was hiding in 2009, and they aren’t now. What they were saying is that there might be ways to use tools of geographic analysis to try and predict where bin Laden might be. In 2009, Gillespie was quoted as saying, "we are all wondering where bin Laden is hiding," Gillespie says. "We just wanted to offer the techniques we have to help." That makes sense, but the question then becomes whether the techniques they offered can help.
They can, but not without some work.
First, the authors assume that there are equal travel costs to all points from Tora Bora, regardless of terrain, cultural geography, political geography, transportation networks, or other features. This simplifies the model but is unrealistic. The model would need to incorporate varying travel costs to be more useful.
And even though their model doesn’t make assumptions about travel costs, the authors do, and they set a somewhat arbitrary limit of travel of 20 kilometers from Tora Bora over an eight year period. This assumption flies in the face of reality, and assumes that the most powerful terrorist leader on the planet could not get a little help from his friends. The model needs to deal with time and resources better to be more useful.
Once the authors isolated a large area of interest, they used Defense Meteorological Satellite Program Operational Linescan System imagery to isolate urban islands. This methodology measures nighttime lights, and basically says "the more light there is at a given point the more people there are likely to be at that point." Using this technique the authors identify 26 urban islands where bin Laden could be, and this more or less works. They then chose Parachinar from this list based on a set of assumptions — which turn out to be questionable.
For example the authors fail to take into account important cultural factors. A 2009 response to the paper by Murtaza Haider notes "Since I am from the Northwest Frontier Province, I find it a little odd that Osama [a Sunni Muslim] may be hiding in the only Shiite majority town in the entire tribal region of Pakistan." The lack of cultural awareness in the paper, and the consequent failure to realize that not all areas would be equally available to bin Laden as hiding spots, weakens the analysis considerably.
Finally, once they had narrowed down their search to Parachinar, the authors used high-resolution satellite imagery from Digital Globe’s QuickBird satellite to perform manual imagery analysis of structures to assess whether they might be potential hideouts. But here at the last mile their methodology breaks down even further.
They score structures based on a profile of bin Laden’s assumed "Life History Characteristics": "Each structure was quantified as 1 or 0 for accommodating each one of these six characteristics. Structure values were summed and the highest values (for example 5 or 6) were selected as his probable location." But many of their assumptions are either false or flawed.
For example, based on the characteristic "Is 6' 4" tall" the authors assume "Tall building." But tall people do not get tall buildings and short people do not get short buildings — all people usually get buildings that are about the same size. Next, the authors assume that bin Laden had kidney problems that required access to a dialysis machine. But as Danger Room’s 2009 review points out, "the professors accept as fact that bin Laden requires a kidney dialysis machine. That means he must need to be close to an electrical grid or generator, the UCLA pair reason. Too bad the thing is complete folklore — debunked again and again."
There are flaws in most of the remaining four assumptions, and what this leads to is a high chance of the model producing false positives. Also, the work in this last step would require hours of skilled human imagery analysis per candidate structure, which means that even to get to the 26 candidates in Parachinar it could take weeks of work — and remember that Parachinar isn’t even close to Abbottābad.
These could be addressed and the model could be refined to produce better results. The authors note that "These methods are repeatable and could easily be updated with new information obtained from the US intelligence community on his last known location." As possible next steps following from their research the authors wrote that the National Security Agency, the National Geospatial Intelligence Agency, and the National Reconnaissance Agency "should also disprove the hypotheses that Osama bin Laden is: (1) located in the Kurram region of Pakistan, (2) located in the city of Parachinar, and (3) at one of the three hypothesized buildings."
But what they failed to realize is something that was proven out in bin Laden’s takedown: none of these agencies could perform these tasks alone. They are all agencies that work remotely, and use technology to do so (and NRO isn’t even really in the analysis business, which the authors miss). But the takedown Osama bin Laden took months of physical surveillance, human intelligence and finally 79 pairs of boots on the ground (and four paws) to assault the compound, kill its defenders, and kill bin Laden. If one had to repeat this process for every candidate turned up by the UCLA team’s things would have gotten ugly. And they will get ugly if we rely on this kind of methodology going forward.
The kind of abstract, data-driven analysis that Gillespie and Agnew propose can be effective as a tool to direct human intelligence and operations, but only if it is combined with a knowledge of human geography. This is a critique that can partly be heard from reports penned by the Army’s controversial Human Terrain project to Major General Michael Flynn’s 2010 paper "Fixing Intelligence: A Blueprint For Making Intelligence Relevant in Afghanistan." But it can never be a substitute for ground truth knowledge and operations. The takedown of Osama bin Laden shows that despite our best technologies there is no magic to fight battles, only hard work.
Satellite image courtesy of GeoEye
