"Smoke clouds are metadata: they hold information regarding both space and time." The voice is calm; the scene playing out on the laptop is not -- intense cobalt-yellow flashes from detonating ordnance precede pillars of black smoke that partially mask the city beneath.
The images and sound come from an online report by Forensic Architecture (FA), a south London-based research organisation whose approach to data collection and analysis is changing the way human rights controversies are investigated, in part by using mobile-phone footage from ordinary people. "The concept of testimony is being completely reformatted," the organisation's director Eyal Weizman tells WIRED. "Usually, human rights organisations have to wait days, even months, and collect things from memory. But these are testimonies of people who were there, technological testimonies through their cameras and videos."
Compiled in partnership with Amnesty International, the media-rich document is called Black Friday -- Carnage in Rafah During 2014 Israel/Gaza Conflict. The strikes are from US-made Mk84 bombs that were dropped by the Israeli Air Force on the al-Tanur district of Rafah in southern Gaza on Friday, August 1, one of the most intense 24 hours in 2014's seven-week war between the Israel Defense Forces (IDF) and the Palestinian group Hamas. Each bomb contained almost one tonne of high explosives.
The FA report is playing on a laptop in the conference suite of the Ambassador Hotel in Arab East Jerusalem's diplomatic quarter. It is August 1, 2015, a year to the day since the air strike, and Weizman is about to publicly launch Black Friday and also the online platform that FA has designed to carry the report, an interactive tool which the group will be making available as open-source software in the hope it will be used by observers of other conflicts around the world.
There is a frisson among the assembled Israeli, Palestinian and international journalists when Weizman -- a compact and bearded man, today wearing chinos with trainers -- enters the room, trailing a wake of officials from Amnesty International and members of his team. Weizman is an academic star: the 45-year-old Israeli is professor of spatial and visual cultures at Goldsmiths, University of London -- home to FA's headquarters -- and is also an global professor at Princeton. As the director of Forensic Architecture, Weizman has invented a new academic discipline, perhaps even a whole new science. "But this is not cold science," he says. "This is committed, engaged, citizen science."
FA investigators take standard architectural and digital tools such as telemetry, video-footage syncing and shadow clocks, and repurpose them to reveal the secrets of conflict zones. Weizman's team has also pioneered plume analysis -- the study of the whirling clouds of mainly atomised concrete created when bombs are dropped in urban areas. Each plume has a distinct shape, like a fingerprint. Weizman and his colleagues' brief is as wide-ranging as man's cruelty to man: they mine the information seam where enemies clash, where migrants drown, natives are dispossessed and civilians bombed.
One of its most notable projects, the "left-to-die boat" case, used FA's mapping skills, survivor reports, mobile-phone records and nautical charts to show how western powers -- present in the area in great naval force -- consistently ignored the pleas of a stricken boat carrying 72 migrants from Libya to Italy, leaving it adrift for two weeks in March 2011.
When possible, the team furnishes prosecutors with evidence to be used in court cases and international tribunals, so that human rights violators can be brought to justice. It has mapped the Guatemalan military's genocidal campaign against indigenous peoples in the 80s, producing an interactive online report. Its work on US drone strikes in the Waziristan province of Pakistan exposed the price paid by civilians for the west's war against Islamic militants in the region and pioneered the use of 3D landscapes as a reconstructive memory chamber for survivors of military attacks. FA's staff are activists as much as investigators and, although they often use existing techniques, they approach them with a fierce, almost political will.
This will comes directly from Weizman. He has written many award-winning books such as Hollow Land and The Conflict Shoreline, with photographer Fazal Sheikh. Weizman's work uses post-modernist cultural theories about politico-physical structures and legal processes to explore the mechanics of conflict, especially the near-century-old conflict in Palestine.
The Israeli-Palestinian conflict aside, it is a strange and unsettling dispute -- where the air above your head is contested and the ground beneath your feet subject to claim and counterclaim. It's a conflict in which the architecture of the region is itself a weapon as much as the Palestinian Qassam rockets or the Israeli Merkava tanks.
The most obvious example of this is the security wall next to the West Bank. Weizman has written about it at length. Israel-Palestine is criss-crossed with other borders and divisions; some are the baffling results of the 1993 and 1995 Oslo Accords which, in the hope of peace, divided the occupied territories into Areas A, B and C. Others, such as the Green Line, mark the end of fighting that took place as long ago as 1948.
Weizman is a product of this contested landscape. "I have lived half my life in the UK," he says. "But my formative years were here. I feel a great love for this place and I know it very well. To operate, politics needs obstruction, it needs the know-how, the love, the familiarity with every stick and stone in the area. Putting your body where your politics is, is very important."
One case particularly influenced Weizman's thinking -- the death in 2009 of Palestinian demonstrator Bassem Abu Rahma in the West Bank village of Bil'in. While demonstrating against the wall that encroaches on the village's land, the 31-year-old was fatally wounded by a tear-gas canister fired over the wall by Israeli forces. To investigate, in conjunction with Situ Research, an architectural studio based in NYC, Weizman utilised what he calls video-to-space analysis. Human rights activists -- often from the Israeli group B'Tselem -- regularly film demonstrations such as those at Bil'in. Three video cameras caught the moment when Abu Rahma, wearing a yellow football strip, was hit in the chest by what FA claims, and claims to have proved, was a deliberately aimed round.
According to the FA report into the death of Abu Rahma, the video footage contains "much spatial information". After syncing the three films of the event by using clues on the soundtracks, the team created a digital model of the area around the wall where the incident took place. They were then able to position all the participants, including Abu Rahma and the Israeli soldiers who shot him. This made it possible to study each moment in the unfolding of the event and even to trace the path of the tear-gas canister's trajectory -- which led to a soldier and a weapon. The report concluded: "the weapon was being aimed... with the likely purpose of killing or maiming the demonstrator."
The FA's accompanying digital evidence was covered by media and, in a rare event, the Israeli military prosecuted one of its men. However, to Weizman's dismay, the case was dropped in September 2013 because of "lack of evidence".
Incidents such as the Bil'in campaign pit Weizman against the Israeli government, which is wary of him. Getting into the country can be testing: his trips are often rushed and each time he doesn't know how difficult security will make things for him. Some of the Israeli reporters in the Ambassador Hotel room ask: what about Hamas? Where is the report about them? Amnesty International has issued a report called Strangling Necks, detailing Hamas's torture, abduction and summary execution of fellow Palestinians during the 2014 war. Nonetheless, some Jewish groups in the region regard Amnesty as biased against Israel. Amnesty and Weizman's team faced death threats before the launch, including the following message: "I plan on visiting you with an M-16 very soon. Consider yourselves warned." Accordingly, three plain-clothes Palestinian security men are concealed among the crowd.
Weizman knows about the death threats, but he is calm when he takes the podium. The wider circumstances of August 1 are generally agreed; it is the specifics that Weizman wants to reveal. The morning had begun optimistically: the UN- and US-brokered 72-hour ceasefire had come into effect and hundreds of civilians were returning to homes they'd fled. They came on foot, by car, by donkey and cart. As they streamed back eastwards along the main Salah al-Din Street route, codenamed Tancher by the IDF, an Israeli air and ground assault was unleashed on al-Tanur.
Both sides contest who broke the ceasefire. The IDF was searching for the tunnels Hamas uses for offensive operations and investigating a Hamas outpost in fields near the border with Israel. In the engagement that followed, one Hamas fighter and two Israeli soldiers were killed. A wounded Israeli, second lieutenant Hadar Goldin, was seized and taken into the tunnel complex.
The IDF responded by implementing the Hannibal Directive. This simply means doing everything possible to prevent a soldier being taken captive - even if that means the soldier's death. An Israeli military inquiry found that more than 2,000 bombs, missiles and shells were fired into Rafah on August 1, including 1,000 in the three hours following the capture. The response would continue for four days. Amnesty estimates civilian deaths in Rafah at between 135 and 200. Goldin was killed as well, although by which side remains uncertain.
Even as the assault was being unleashed, it was being documented and analysed. Across Rafah, Palestinians, human rights volunteers and medical workers were reaching for their smartphones; on rooftops, news organisations were training their cameras on the bomb bursts; and 694 kilometres above, the French Pleiades satellite was passing over Gaza.
It meant investigators were able to collate a vast amount of data. According to Amnesty, the FA team had access to "testimonies from victims and witnesses; reports by human rights and other organisations; news and media feeds, public statements and other information from Israeli and Palestinian official sources; and videos and photographs". By using shadow clocks and hi-res images of tank tracks left over the four days of fighting around al-Tanur, they were also able to let the land itself tell the story.
Four ways in which Forensic Architecture, with SITU Research, is shedding light on conflict
"For us, the question is how the city senses war," Weizmans explains from his office at Goldsmiths in Deptford. "A war starts; tens of thousands of foreign people enter into a familiar city that has hundreds of thousands of people in it and at that moment everything starts recording. People's memory records, the grass records, the trees record, the plumes in the air record, the concrete records. Everything is recording in a variable way. But these sensors are weak, fucked-up sensors. You need to develop ways of interpreting and reading and mediating those things."
By analysing smoke plumes, FA investigators say they could determine exactly where the IDF dropped Mk84s. By studying bomb craters and the tracks of bulldozers and signs of digging, they concluded that in order to end Goldin's imprisonment the IDF set out to destroy both the tunnel networks and any possible escape route via Tancher.
As well as Palestinian witnesses, the Black Friday report includes the testimony of Israeli soldiers collected by the veterans' organisation Breaking the Silence. "After the area was hit by 1,000 shells that Friday morning, I saw Tancher in ruins," one shocked soldier said of the destruction. "Everything was totally wrecked."
The previous cycle of violence in Gaza in 2008-2009 also had many civilian casualties. But in that conflict the deaths of 900 Palestinians and three Israelis were less publicised at the time than those in 2014. Weizman points to one key difference: "Social media," he says. "In 2008-9 all the expertise, all the testimonies, were after the fact. In the five years since then a flood of images has opened up, there is much more stuff from the ground. And also a change in satellite images. Previously we had access to US satellite images and an act of Congress meant all US images taken over Israel were pixellated. But in 2014, there was a French satellite above Gaza, and they're not regulated by that. So for the first time the war was captured in high-resolution images."
During WIRED's first meeting with Weizman and his Goldsmiths team they were looking at those images. One of the analysts, 26-year-old Greek architecture graduate Christina Varvia, was cross-checking the video footage against satellite images on her desktop. "We were very lucky that this particular strike that you are seeing here has actually been recorded by the satellite," she says. "The satellite has very accurate metadata - this picture was at 11.39am."
One still from al-Tanur showed two bombs in mid-air. "The material is quite harsh," Varvia says, "but part of the reason we do this is because we understand the heaviness of it."
Weizman acknowledges the contributions of the witnesses. "Sometimes they risk their lives to take those images," he says. "You hear the distress of people; they say things as they record and see people dying around them. These are valuable real-time testimonies that not only reflect what happened but the state of mind of people right at the moment it happened. Our way of valuing that is to listen to all their bytes and pixels, to find whatever is hidden in them."
In August 2015, Weizman drives WIRED through East Jerusalem to the Old City. We pass the 15-metre-high Ottoman walls, which are still imposing, despite the highway that runs beneath them. Weizman gleefully explains the Arkansas Big Mac -- US President Bill Clinton's surreal, in retrospect, plan to portion out the space occupied by the remains of the biblical Jewish temple and Islam's Haram al-Sharif by horizontally dividing up the air, the Earth's surface and the territory beneath it. An internal border would have run under the floor of the Haram al-Sharif, buffered by a 1.5m-thick UN zone and then Israeli territory.
The windscreens of tourist coaches on the Mount of Olives glint as Weizman continues down into the Kidron Valley, through contested scrub the Israelis claim was King David's city, and enter the Arab township of Silwan. The community is under constant pressure from settler groups who believe it is the right and duty of Jewish people to live here and Israeli archaeologists have been digging beneath Silwan's margins in search of signs of an ancient Hebrew capital. Here in the tumbledown vertiginous closeness of Silwan's housing blocks, Weizman explains how to read a landscape like a forensic architect. He notes where the road has been cracked by the subsidence caused by the excavation as he halts on the much-repaired Tarmac. "The dips in the road, the bumps, every crack -- they are all witnesses," he says. "The idea is to use forensic architecture as a method that extends deep into the facts and looks at them and maps them out to see the materialisation of political forces. Forensic Architecture assumes that every bit of material reality is the product of a complex force field that extends in space and time. So you can take an inanimate object and see into it, almost like a crystal ball."
No crystal ball is required in the landscape south of Jerusalem; the signs of conflict are all around us. The built landscape is an Escher drawing: Israeli-only roads go through tunnels driven under occupied mountains; the Jewish settlement of Gilo looks across the valley to Palestinian Beit Jala, the highway between them supported on Israeli concrete pillars that stand on occupied ground.
Entering the eight-lane checkpoint that marks the beginning of the West Bank proper, Weizman explains the dog-legs and U-turns he must perform to get round the checkpoint system and drive east into Beit Sahour, a Palestinian village outside the West Bank city of Bethlehem. He was a founding member of the architectural collective DAAR here and still keeps an office.
But today he is turning right off the highway and heading west into fully Palestinian-controlled territory. An ominous red signpost warns: "This road leads to area a under the Palestinian Authority. The entrance for Israeli citizens is forbidden, dangerous to your life and is against Israeli law."
Weizman shrugs at this instruction from his mother country. He has been discounting strictures since he was a teen embarking on the military service which all of Israel's Jewish citizens -- with the exception of the ultra-religious -- must complete.
Weizman would achieve a remarkable victory at Battir, a Palestinian village he describes as "a small jewel, and a tiny garden". Battir sits right on the 1967 Green Line that most international organisations envisage as the eventual border if there is ever to be a two-state solution. Driving through Battir's outskirts we pass the Christmas cake-white mansions of rich Palestinians -- one of which is due to be demolished because its footprint strays out of Area B, joint Palestinian control, into Area C, Israeli control, a case Weizman has worked on but fears he has lost. Twitchy guards patrol inside the high iron fence, awaiting the arrival of IDF bulldozers.
Beyond that the road slopes down to reveal the main village perched above a hillside divided into 3,000-year-old agricultural terraces. The Roman cistern that feeds the terrace irrigation system of open channels and sluices is the oldest in the Holy Land. This agricultural and archaeological resource came under threat when the Israeli army decided it was going to run the separation wall along the border here. The security road that runs alongside would have destroyed the lowest terraces.
Smarting from the events at Bil'in, Weizman decided to change tack. "It had been an error entering into a debate with the Israeli military in an Israeli court arguing for human rights," he says. "But Battir was not a claim about rights; it was a claim on behalf of the landscape. Of archaeology, of nature."
To illustrate what he means he walks along the Green Line, the 1967 border marked here by the old Jaffa-Jerusalem railway. Weizman picks his way through terraces planted with vegetables and fruit trees. Across the valley on the Israeli side there is no sign of habitation and the hills are covered in conifers. "That is how you know that side is 1948 Israel and this side Palestine," says Weizman. "The tree planting."
Weizman shows how the lower terraces would have disappeared if the wall had been driven through here. FA used digital modelling to illustrate the effect of the proposed route of the wall through Battir. "Members of the group had previously submitted the file for UNESCO listing. We added our evidence to this, taking an oral history of the land from the farmers we built a picture of what was going to happen. There was even testimony from Israeli settlers who didn't want to see the terraces demolished." UNESCO gave Battir the listing, putting pressure on the Israeli high court to stop the wall. In January this year it was finally decreed that no wall would be built at Battir.
Here, at the scene of that judgement, Weizman's previously scholarly manner finally slips. "It was a landmark case," he says as we look down on the terrace. "There will be no wall here. There will be a permanent gap in the wall. It is a small victory and the danger is successes like this make the process more acceptable and allow the bulldozers to go in elsewhere but, yes, it felt good."
Not content to be the man who stopped the wall, Weizman wishes to use the precedent he established at Battir to stop all wall building -- to move, in his view, towards a just settlement of the conflict. "It was that terrace that won the case," he says. "We learned that human rights take you nowhere in this area, but if you claim on behalf of mountains you have a case. Battir was saved because of its natural beauty, historical significance and religious significance," he says. "I would claim that is the case with the whole of Palestine."
It wasn't just 21st-century technology that won the day. Villagers could give Weizman an oral history of the land they lived on, tell him what grew in each terrace and how long it had grown there. In building his digital picture of the landscape Weizman was able to plug into Battir's collective knowledge, a folk memory that goes back to a time before modern borders or states. Each farmer knows the history of each tree, the planting cycle of each terrace. "We understand the relationship between memory, architecture and violence," Weizman says. "Take the woman who survived the drone strike in Waziristan. She was very traumatised; she lost relatives in there. We returned her digitally to the site of the attack and built it together with her, reconstructed her family house that had been hit by the drone. During the modelling process she was meticulous about every window, every object we placed in there, every person. But she was very obsessed with a fan. In the beginning she said it was on the ceiling. Then she said no, it was a standing fan. She asked us to move it to the left and then to the right and then back again, until we were wondering, what is it about the fan? But when we made her walk through the space she recollected exactly where it had been, and that after the strike had killed her family she had found bits of human flesh on the blades of the fan. You see, the fan acted as an anchor for her memory and in the end we reassembled that memory in a digital space."
Later in the day Weizman drives out towards the towers of Tel Aviv. Presently the traffic brings him to a halt as if he were in a city like London or New York, where everyone has sophisticated technology in their pockets and, thanks to social media, the chance to record and share what they see. Weizman has found a way to harness these, our everyday digital diversions, for a fierce, moral purpose. "With forensic architecture you work at the micro but your eyes are on the macro," he says. "Whether it's the terraces at Battir, the cracks in the roads of Silwan or the bombs in Rafah -- whatever it is, you fight over the last rock."
Michael Hodges wrote about North Korea in WIRED 04.15
The Digital Analyst
Christina Varvia, Forensic Architecture researcher
"I analyse the available sources, extracting material and then making sure that this analysis can be visualised in a way that communicates well.
"With each image we try to find the location and time so that we can build a system and understand exactly where and when a bomb fell. We do this through a series of techniques. We locate a piece of footage -- for example, the video of the al-Tanur strike -- and we try to figure out where the photographer was sitting. We are basically identifying certain sites we can find on the footage. Then we mathematically analyse the perspective to understand the view frame of the camera -- what is it framing? We do this because in every other process we need an exact location of where it is. We need to know the exact orientation of the footage. It's a process of measuring the buildings' distance from the photographer, treating it as an elevation, testing it as a drawing and then transferring that.
"My colleague Gustav is producing animations and graphic material for our reports on the Gaza and Rafah projects. He is working on a way of using the photos to construct 3D models from them. Something like this exists already, as photogrammetry. This is a different kind of process that we're using with software and animation. It uses existing architectural software but we are re-appropriating the tools in a way that's closer to animation -- creating 3D objects and placing them in footage. We are pushing the tool to its limits."
THE PLATFORM SPECIALIST
Francesco Sebregondi, Forensic Architecture researcher, project architect, PATTRN
Based at Goldsmiths, University of London, Francesco Sebregondi is a researcher and project architect of PATTRN, a new open-source app for data visualisation and mapping.
"The tool facilitates the mapping of complex events based on different data sources -- it's a form of data-driven, participatory fact mapping. Increasingly in the cases we work on with Forensic Architecture, we are dealing with vast amounts of data fragments - reports of events, photographs, videos, social-media posts and satellite imagery. We realised that we needed a platform to compile all this data, cross-reference it and start to understand the relation and hidden connections between dispersed events.
"So we developed PATTRN, which works as an online visualisation platform that enables the user to explore all the data compiled in a database. People can directly submit data to the platform, so that the big picture of an ongoing event can be built collaboratively.
"With the global spread of smartphones and social media, conflicts and protests around the world are increasingly reported by the very people that experience them first- hand. Syria, Ukraine or the Ferguson riots, are all cases in point. But it's also difficult to make sense of this mass of data, and to distinguish between the facts and rumours. PATTRN sets out to address these new conditions, and to support more transparent and citizen-driven information about these complex events."
This article was originally published by WIRED UK
