Gloomy predictions abound that the applications of artificial intelligence and machine learning will put huge numbers of people out of work in the coming years. But the corollary is that these technologies create opportunities to develop new goods and services that will bring new jobs. What's certain is that advanced implementations of computer science are beginning to disrupt our lives. We must start thinking about how these technologies are applied and regulated if we are to reap the benefits and minimise potential harms. Read more: From nuclear war to rogue AI, the top 10 threats facing civilisation
The introduction of the steam engine in the 18th century disrupted the life of the agricultural labourer and fuelled the rise of cities, creating new industries and new jobs. Traditional professions such as medicine and law were largely unchanged. But the latest industrial revolution has the potential to change almost every form of work.
So far, most of the applications of machine learning appear largely benign, but they are having important impacts, which most of us are barely aware of. How often is your credit or debit card stopped or queried? This used to happen frequently, but is becoming much less common for most of us. This is because of the application of machine learning about our financial behaviours and patterns of movement. Banks and credit agencies learn continuously about the purchases we make. This is convenient and diminishes the risk of theft. It also means that banks can know more about our lifestyle than our close relatives.
Marketing and consumer-service companies learn continuously from what we buy online, both as individuals and en masse. Store cards have generated vast amounts of data on our shopping habits that influence almost every aspect of the design and placement of products in stores and supermarkets. Our Google search results and email inboxes are influenced by our prior online behaviour. Is it helpful that Netflix or Amazon recommend movies that we would like to watch, or is it an invasion of privacy? Do we know who is actually accountable for these recommendations? Do we even care?
When we buy online, what does it mean to be a "good" customer? Are we rewarded or punished by sellers who can know more about our psychological traits than we do ourselves? We think that being a good customer should correlate with getting a good price. But to a seller, a good customer is one who is known to want a particular product, and that may be correlated with willingness to pay more. So machine learning can enable a seller to charge us more, not less, for some goods that we buy regularly. It can work out whether we are a sucker, for whom the price of particular goods is elastic - and then stretch that elastic to the full.
There is nothing new in dynamic pricing - just compare the prices of trips taken in school holidays as opposed to term time. There can be important benefits from the use of algorithms to generate dynamic pricing for goods and services. When demand for energy rises, it makes sense to charge more to discourage unnecessary use of gas or electricity and to limit the need for additional generation capacity. Uber dynamically increases prices for travellers when demand is high and drivers are in short supply. That acts as an incentive to get more drivers on the road in places when they are needed most. But equally, if there is an emergency such as a storm causing public-transport systems to fail, a large hike in Uber prices may be interpreted as an attempt to gouge the consumer. Some of the most dramatic impacts from AI and machine learning may be felt in the stuffy cloisters of the most traditional professions. Medicine and law have shown a high degree of immunity to disruption. While most service industries have been transformed over the past 20 years by IT, these professions have been largely resistant to change.
Why is this? These professions typically require long training and apprenticeships. They rely on specialised skills and the application of judgement for which the professional can be held personally accountable. But that begs an important question - how are professionals held accountable and who judges the quality of their skills and judgement? The traditional answer is that only an equivalent professional is capable of judging their peers. This makes sense but is also problematic. It has underpinned the development of professional regulatory bodies that typically have a strong element of self-regulation. These bodies can also create restrictive trade practices and enable a culture of professional elitism and deeply entrenched conservatism.
There is a particular set of values commonly associated with being professional. Experience, expertise, trustworthiness, wisdom and good judgement are all attributes aspired to by senior professional people, be they doctors, engineers, lawyers, civil servants or the clergy. What lies behind these attributes is knowledge and values, such as "do no harm" in the case of medical practice. So medicine is practised in the context of "evidence" and law in the context of legislation coupled with "case law".
But the challenge to the medical and legal professions is that much of their work could be dramatically improved and, in some cases, substituted by computers that learn and apply AI. If judgement is an essential attribute of a professional, is it possible for a computer to do the same thing? Can we distinguish different types of judgement and decide whether a human and a computer are capable of making these?
In medicine, one of the most important skills requiring judgement is the ability to diagnose and treat, frequently in the face of uncertainty, due to incomplete evidence. But in the case of cancer, computers are starting to ingest huge databases of scientific evidence. These can be coupled with huge numbers of individual medical case histories associated with genome data about the cancer and clinical responses to therapy. What patient would not want their treatment to be based on the best evidence? On this basis, is this the end of the medical oncologist?
Medicine is not a pure science. The "art and mystery" of medicine, in essence, the second area of judgement for the physician, is to provide advice. Advice on when not to treat, on when the treatment may be worse than the disease, and guidance for patients on a morass of competing therapies. Some of these may make them feel better, even when they have no impact on the progression of the disease. Could AI achieve this - and would we want it to?
What about the law? Would you rather be found or guilty or not guilty by a judge, a jury or a computer? The test of guilt applied by criminal courts is "beyond reasonable doubt". But what is reasonable to one person may not be reasonable to another. And the test in a civil court of "on the balance of probabilities" is much more nuanced.
Computers are good at working out the balance of probabilities. Their spinning discs can accrue huge databases of criminals; their processors can work out the chances of matching a DNA sequence or a fingerprint. And we have started to leave an extensive trail of digital evidence as our movements are recorded and geolocated from the array of wireless and GPS-enabled devices that we carry. The probabilities associated with many pieces of evidence can be linked. Concepts such as "beyond reasonable doubt" or "on the balance of probabilities" can be quantified in principle - let's say 99 out of 100 for the former and 51 out of 100 for the latter. So these concepts are an area of human judgement where the computer should have a lot to offer.
What about sentencing? Would you prefer to be sentenced by a judge or a computer? Most of us would probably mentally calculate which route would be likely to result in the least painful sentence. That calculation might result in a different outcome depending on your gender, race and social and economic status, since sentencing by humans can be systematically biased according to these and other factors.
Indeed, there are many circumstances in which humans show systematic bias. There is evidence that individual judges and doctors behave in systematically different ways - the identity of the judge sentencing you or the doctor treating you may result in significantly different outcomes. But it is not only judges and doctors who show systematic bias - there are biases at every stage in the criminal justice process and in the provision of clinical care. So the probability of being stopped and searched, the likelihood of being jailed or bailed, the sympathy of lawyers towards their clients and the attitudes of jurors to the accused are influenced by several factors. These are sometimes weakly correlated with the guilt or innocence of the accused. In a similar fashion, your social circumstances, your GP, your nurse and many other factors systematically influence your medical outcome.
When it comes to sentencing, spinning hard discs can contain a complete record of previous sentences. However, machine learning could internalise all of the implicit biases contained within the history of sentencing or medical treatment - and externalise these through their algorithms. An example of this happening in advertising was the discovery that a search on Google for "CEO" returned images of almost exclusively white men and that it delivered far fewer adverts for high-paying executive jobs to women than to men.
One solution is to never elevate the recommendation of a computer above the judgement of a human. This places the computer in an advisory role. But what would happen in the case of a bad outcome? In the case of a failed cancer treatment, would the doctor who rejected the treatment recommendation of the computer get into trouble - or would the one who followed the recommendation? And, ultimately, would the computer end up holding the top trump by default, as it is likely that strong human cognitive biases will develop against ignoring recommendations by computers?
When it comes to subjective judgements, will we move to a world where professional norms and values will be built into computer algorithms? In the case of judicial sentencing, we could deliberately build in a set of biases. Weighted rules could be created to take into account - or ignore - a criminal's difficult childhood. Sentencing could be weighted according to the accused's family circumstances. It could also be designed to ignore these circumstances. In short, looking through the eyes of a human, sentencing by computer could appear to be sympathetic or unsympathetic to the specific circumstances of either the perpetrator or victim - according to one's particular biases.
So, if we are to get the most out of AI and machine learning, we will need to work out mechanisms to understand the operations of algorithms, in particular those that have evolved within a computer system's software through machine learning. Do we really know how AlphaGo, DeepMind's computer program, won the series of Go matches against a human world champion?
It is hard enough to hold humans to account for their judgements, partly because we have only the vaguest understanding of the cognitive mechanisms that result in human judgements. We do our best, by using other "wise" humans as judges. But when it comes to machine learning, the answer is less obvious.
A computer, by definition, cannot be held accountable for anything because there is no mechanism to hold it to account, short of turning off the electricity supply or destroying the hardware. Only humans can be accountable. In the case of a computer, who would be held accountable - the programmer, the supplier or the end user? Given the difficulty in knowing the exact nature of an algorithm that evolves as the machine learns, it might be impossible to completely explain its operation. So, ultimately, is it a case of caveat emptor? In this model, the computer advises but the user decides. But how can this work in an autonomous system - a car, train or plane? One solution would be to hold human programmers strictly accountable for the impacts of their programming. But that could be so draconian an accountability that no one would take the risk of programming an algorithm for public use, which could deny us the benefits of machine learning. It is not as though humans are perfect when it comes to making life-and-death decisions, whether it involves flying a plane, engineering a bridge or treating cancer. Why would one trust a human with only partial knowledge of the evidence on a life-or-death issue? Read more: Machine learning versus AI: what's the difference?
What about the potential for machines to judge machines? This already happens with safety-critical computer systems where parallel systems can take over when one computer fails. By learning as a fleet of cars, Tesla already collectively raises the standards of a whole fleet of cars in a way that humans could not. We may need machines holding other machines to account in some circumstances. Already, high-speed, high-volume algorithmic financial trading relies on code to examine and regulate the vast volumes of trading that cannot be encompassed by humans. So-called "reg tech" is needed as an adjunct to human regulators in the world of big data, AI and machine learning.
This is an urgent problem to tackle. Partnerships are being announced between healthcare providers and companies developing and selling healthcare advisory systems built around algorithms that are developed by machine learning. How are these systems to be organised in the world of highly regulated medicines, diagnostics, devices and therapies? Medicine regulators, working to similar protocols around the world, license drugs and place strict product liability on their manufacturers. But we know the chemical structure of medicines and the precise specification of medical devices. We can't know the precise structure and workings of algorithms that evolve continuously by a process of machine learning. International regulatory processes demand that medical algorithms are regulated as though they are devices - but what does this mean in reality?
These problems should not be insurmountable. We may know the chemical structure of medicinal drugs, but we frequently have a very incomplete understanding of how they work. Drug regulation is guided by evidence of efficacy traded off against side effects, in relation to the severity of the disease and the benefits of the drug. An algorithm trained by machine learning differs from a drug in that its structure changes as it learns. The implication is that regulation needs to be more continuous, examining effects of the algorithm as it is used and evolves in clinical practice. Read more: Holding AI to account: will algorithms ever be free from bias if they're created by humans?
Medicine and law have shown a high degree of immunity to disruption, but some of the most dramatic impacts from AI and machine learning may be felt in the stuffy cloisters of the most traditional professions
But there are strong arguments that drug regulation should head in the same direction, as increasing amounts of information become available about the effects and side effects of drugs as they are used in large numbers of patients. Surely this should be a benefit of modern IT applied to health systems.
If we are to apply algorithms for purposes that could be associated with harm to humans, we need to learn how to trust them. What crash rate might be acceptable for vehicles driven by computer systems? At one level, any rate associated with less human suffering than vehicles driven by humans would be an improvement on the present. Yet there would be uproar at the killing by autonomous vehicles of bystander pedestrians, even if this was an extremely rare event.
We might consider that algorithms should be held to much higher standards than we imperfect humans. Ultimately, because algorithms impinge on the judgements made by professionals, they affect important issues about ethics and values. How should these be factored into regulation? One useful metaphor might be the approach that was applied in the UK to the development and use of technologies applied to human embryos. Here, the application of science meets strongly held human values head-on.
Consideration by the Warnock Committee led to the creation of the Human Fertilisation and Embryology Authority. This is a regulator that has a duty to consider emerging science, to engage in public debate and to discuss contentious issues with the legislators and policymakers who decide the framework for application of emerging technologies. Do we need something similar in the area of machine learning and AI? This issue is being considered in the UK by a joint working party from the Royal Society and the British Academy.
A key role of government is to maximise the health, wellbeing and security of citizens. So it needs to consider how to maximise the benefits from these technologies and to minimise their potential harms. This will require intelligent regulatory approaches. It will also require businesses to think very carefully about how they apply these new technologies. This is an area where ethical consideration - and ethical implementation - really matters.
This article was originally published by WIRED UK
