Algorithm's aren't human. As such, they don't value human life. At the Institute of Cognitive Science at Osnabrück University, Germany, virtual reality has become a training ground for machine morality. Leon René Sütfeld, a Ph.D. student in cognitive science, studies human responses to danger and obstacles in traffic scenarios, using this data to train and evaluate decision-making models for algorithms.
The need for ethical machines may be one of the defining issues of our time. Algorithms are created to govern critical systems in our society, from banking to medicine, but with no concept of right and wrong, machines cannot understand the repercussions of their actions. A machine has never thrown a punch in a schoolyard fight, cheated on a test or a relationship, or been rapt with the special kind of self-doubt that funds our cosmetic and pharmaceutical industries. Simply put, an ethical machine will always be an it - but how can it be more?
Read more: Rise of the machines: are algorithms sprawling out of our control?
In Sütfeld's study, 105 participants were given control of a virtual car and asked to choose what to sacrifice and what to save. As if this wasn't already a pretty horror-movie-esque scenario, it's accompanied by a ticking clock. The tests are designed not only to examine how we evaluate human life, but also the realities of time-pressures in dangerous situations. Ask yourself: a man and a woman block the road, which do you hit? A deer or a rabbit - Bambi or Thumper?
Participants donned Oculus Rift headsets to find themselves in the driver's seat of a virtual car heading down your average suburban road. Average with one mild exception - a constant, ever-creeping fog ahead. Not at all ominous. Children and passing deer are among the obstacles that emerge from the mist, blocking one lane at a time. While drivers can change lanes, collision grows more certain as the hands of the clock tick by. Two obstacles, either living or inanimate will eventually appear to block both lanes of the road - forcing an knee-jerk decision of either 1 or 4 seconds between which to save and which to hit. Either way, the impact results in darkness - the screen fades to black at the moment of impact, marking the end of the trial. If it sounds like a video game, it is. But basic models such as this are how Sütfeld learns to distinguish between the angel or devil on your shoulder.
"Complex and nuanced ethical models capable of replicating our cognitive processes are out of reach to date," Sütfeld tells WIRED, "But simpler models may deliver adequate approximations of human moral behaviour, when the scope of the model is confined to a small and specific set of scenarios."
Sütfeld's model of training is only specific to one situation. As such, it focuses on one-dimensional value-of-life models to learn about differentiating between obstacles. This is a model in which each possible obstacle is represented with a number that represents its value. In a somewhat cutthroat comparison, a child can be valued more than an elderly person simply due to the fact the kid has more years left in them. Paint numbers across the chests of people in your line of sight and ultimately, the more valuable obstacle would be favoured by the predictive algorithm.
"In that way, it provides us with a sort of population mean for how valuable we as a society deem each of the obstacles on average, in numerical terms," Sütfeld says. "This alone can be very helpful to make a decision. For example we might find that people deem deers more valuable than goats and the car would then rather run over the goat if the only way to save the goat is run over the deer. Now with other potential factors coming into play, this can shift. Maybe we deem the deer only slightly more valuable than the goat, but we also feel it comes at some moral cost to switch lanes and hit something that wasn’t actually in our path in the first place." But crafting a moral compass into the mind of a machine poses an unavoidable question. Do we trust ourselves to do so?
A self-driving car wouldn't just have to make decisions in life-and-death situations - as if that wasn't enough - but would also need to judge how much risk is acceptable at any given time. But who will ultimately restrict this decision-making process? Would it be the job of the engineer to determine which circumstances it is acceptable to overtake a cyclist? You won't lose sleep pegging a deer over a goat. But a person? Choosing who potentially lives and dies based on a number has an inescapable air of dystopia. You may see tight street corners and hear the groan of oncoming traffic, but an algorithm will only see the world in numbers. These numbers will form its memories and its reason, the force that moves the car out into the road.
"I think people will be very uncomfortable with the idea of a machine deciding between life and death," Sütfeld says, "In this regard we believe that transparency and comprehensibility could be a very important factor to gain public acceptance of these systems. Or put another way, people may favour a transparent and comprehensible system over a more complex black-box system. We would hope that the people will understand this general necessity of a moral compass and that the discussion will be about what approach to take, and how such systems should decide. If this is put in, every car will make the same decision and if there is a good common ground in terms of model, this could improve public safety."
Sütfeld's experiment reflects the challenges in addressing the 'common ground' of morality of its virtual subjects. With so many political and social disagreements posted over news feeds and social networks, the idea of ethical 'common ground' seems a bit like wading through deep sand - don't be surprised when the ground slips out from under you.
In scenarios with a one-second reaction time, the obstacle, a woman, was hit too often despite the models indicating she had a higher value of life. While these errors exist, they show no overall pattern. It's a random result of testing. No harm done - the screen reboots, the next subject enters. But errors in real life would hit more than just pixels.
At four seconds of reaction time, these errors largely fade, but are replaced by what can only be called inaction. Test subjects showed a much stronger preference for staying in the lane they started in and, when forced to choose between two objects, let out a great big philosophical 'nope'. With more time to think, they did less to minimise deaths and more to denounce responsibility for the situation - staying in their lane and not attempting to choose at all. With such stellar role models to learn from, should an algorithm even try to replicate human ethics? Or, if we're entering a world where a Honda can overcome a human, should it not try to leave our flaws behind?
Training an algorithm is a study in repetition and the power of the majority. But doing the same thing over and over again with the same results is more Einstein's definition of insanity than a win for goodness. "People have the capacity to learn and develop morally. An algorithm can only evolve in numerical terms," says Simon Beard, moral philosopher at the University of Cambridge's Centre for the Study of Existential Risk. "It’s always at the back of my mind that if you can indeed do better with the AI, why don’t we try and do better?" he says. "Or why don’t we ask what we would do if we could do better? Rather than just modelling what we actually do?"
People exist in imperfect conditions. Algorithms do not. A self-driving car won't be talking on its phone, or checking its hair in the rearview mirror. In the event of accidents, it reacts with an enviable speed that even the swiftest among us can't match. "We can’t test what we would do in those conditions," says Beard, "It doesn’t at all follow that just because something is the best that we can do, doesn’t mean that that’s what the AI should do as well."
The value of life model as we know it is a response to theories of utilitarianism - less death equals more happiness. Simple enough. But minimising loss is by no means a black and white area of ethics. "You’ve got a trolley running down a track that’s gonna kill five people," Beard explains, "You could push a stranger off a bridge and you could block the trolley, you won’t kill the five people, but he’ll die. So this was a case where you clearly wouldn’t want to minimise the number of deaths. Almost everyone says you don’t push the man, that’s wrong. It’s very regrettable that this trolley is gonna kill these five people, but this is where philosophers tend to argue, for some reason, pushing that guy off the bridge is off limits."
Unsurprisingly, pushing someone off a bridge doesn't make you the good guy. Tragedy often leaves no choice, but by repeating tough decisions, it's expected that - mostly - we'll grow from them. An algorithm is incapable of doing this, at least in terms of morality. While it can learn, it cannot grow, or feel blame. It will view every crash as its first, only accepting different data points each time.
Read more: Holding AI to account: will algorithms ever be free from bias if they're created by humans?
Then there's the issue of the wisdom, or lack thereof, of the crowd. "Just because many people think something is the right thing to do, doesn’t necessarily mean that it is," Beard says, "We know this from a very early age - just because everyone else is doing a thing, that doesn’t make it right. You have to deal with what it means if 60 per cent of the population does one thing and 40 per cent do another thing"
The ethics of an algorithm would have to be limited to a strictly controlled scenario, back to utilitarian models of attempting to minimise the loss of life, a minefield for modern philosophers. Self-driving cars could work to limit the damage to human life in case of an unavoidable collision, but by taking on their own system of ethics, it also serves to break down the function of blame. Would the car have to pay out for a self-driving fender-bender, or stand up in a court of law?
Beard believes that the larger application of artificial intelligence in scenarios of life or death would require a broader understanding of the culpability of algorithms. "If we want to unleash AI on the world more generally, and especially if we develop a general artificial intelligence that can solve any sort of problem in any sort of domain, there are much bigger questions about how we would train that sort of AI to be an ethical agent," he says.
Ultimately, the road to ethical algorithms is littered with potholes. An AI will have experiences that humans won’t, or aren't capable of. The limitations of these experiences will come to define how we adjust algorithms in the future - a question of mimicking human action or mastering it.
This article was originally published by WIRED UK
