For all that we have learned about—and from—artificial intelligence over its approximately 60-year developmental lifespan, one foundational piece of our relationship with it has often remained out of reach: a deep human understanding of how complex machine learning models think at a global level.
Dr. Jan Ittner, Chief IP Officer of Data Science at BCG, may finally have an answer.
Enter FACET, a BCG GAMMA-built open-source solution designed to provide accurate global explanations of machine learning models. Developed by Ittner in response to his clients’ desire to better understand and optimize their production processes, FACET was built to empower data scientists to go a step beyond the local perspective of just a single prediction—rather, to help them finally understand and assess the dependencies and interactions of all of the variables within a “black-box” model as a whole. For the first time, scientists have an accessible tool to help machine learning models explain what’s going on behind their own curtain at a global level.
“I still hear statisticians and data scientists say—not being aware of FACET—that the kind of stuff that it does is not doable,” Ittner said.
Thanks to Ittner and his team, it now is. Launched in collaboration with scikit-learn, a popular machine learning library for Python, FACET is built in the model of game theory—a nearly 100-year-old field of study on how individual factors combine to affect an end outcome that has been drawing increasing interest from AI researchers in recent years.
"If you think of a game with players, and there's five players in the game ... and they all make choices and jointly contribute to an outcome, how should they (each) get credit for that outcome?," Ittner said. "Basically the main question is, if you remove players from the game, how would the outcome improve or deteriorate?"
Historically, there’s no shortage of different approaches to answering that one question. But Ittner found the inspiration for FACET in SHAP (SHapley Additive exPlanations), a recent and popular game theoretic approach for explaining the outputs of a machine learning model.
“The brilliance of the SHAP approach is, we just say the different variables are the players, and the prediction is the game,” Ittner said. “So, I ask the model to make a prediction—how will this prediction change after removing one or more of these variables?”
But explaining a model’s general behavior beyond its individual predictions—and the constellation of complex interactions between variables—was simply not workable without a usable algorithm. Determining the dependencies and interactions between a hand-picked set of variables used to be a manual and exploratory process. Now FACET can do it quickly and exhaustively—all at the push of a button. Borrowing from the SHAP approach, FACET evaluates thousands of individual predictions to identify the relationships a model discovers between variables during its training.
"There's always been this trade off—either you have a simplistic model ... and you kind of understand what it does, or you have a 'black-box' model that's very powerful, but is so complicated that no human can understand it," Ittner said. "And that's where FACET goes—it takes a complex ‘black-box’ model, but explains it in a way that is accessible to data scientists."
Shifting from predicting to explaining
FACET’s business impact is significant for both Ittner and his clients—it means they can finally address common questions of how a machine learning model is making its predictions. But FACET’s full impact goes far beyond just BCG. Indeed, it’s arrival could help define the data science industry’s new frontier—using AI to explain, rather than just predict.
“As a general trend, I think the emergence of explainable AI technology such as FACET will shift the focus from ‘using AI to predict’ to ‘using AI to explain’—training predictive models with the end goal of dissecting them and extracting the theories the models have developed about the real world,” Ittner said.
That’s a pretty profound difference. Consider the pharmaceutical industry, for example, where a drug manufacturing process requires thousands of parameters for raw materials, equipment configuration, and process execution. Traditionally, a machine learning algorithm that assesses such variables would be used to simply predict a manufacturing process outcome (like quality or yield). But with FACET, a model can function more like a tool for virtual experimentation, identifying the interplay of process parameters so the manufacturing process as a whole can be understood—and optimized—through informed simulations. In other words, it’s the leap from “your yield will be X” to “this is what you need to do to produce yield X”.
“It was always about, ‘How accurately does a model predict an outage?’, or ‘How accurately can it predict a consumer’s preferences’?,” Ittner said. “I’m saying in many cases we won’t even use (the models) to predict—the real change is, I train a model, and then I go dissect it and I never intend to use it to make predictions.”
BCG has already seen early adoption amongst its manufacturing clients. That adoption is likely to expand in 2021: equipped with this kind of global-level insights, data scientists across industries and use cases can turn to machine learning models to understand and improve real-world commercial, societal, and ethical outcomes in ways never before possible.
Commercial
From manufacturing to shipping, FACET’s commercial use cases feel limitless. Supply chain optimizations are easy targets, where data scientists can more efficiently identify systematic factors that contribute to manufacturing delays, poor product quality, or stock outages. The sky is probably the limit for other economic applications. Fashion manufacturers can more easily justify the production of net-new products when they have a firm understanding of why they are likely to succeed in market. Retailers can optimize their promotion and assortment strategies based on a better understanding of what drives customer demand in different locations and seasons. Sports franchises could further refine the combinations of dietary, training, and rest time factors that lead to peak athletic performance. BCG is even optimistic that FACET could still have a positive effect on the manufacturing efficiency of select COVID-19 vaccines.
Societal
Understanding the near-infinite number of variables that affect public health outcomes from one community—much less one individual—to the next has long challenged policy makers and researchers. But designing truly effective policies or interventions to combat, say, childhood obesity or skin cancer, requires a full grasp of the interrelatedness of all the societal variables in play—it’s the difference between addressing the actual problem, versus just a general view of the problem. FACET solves precisely that issue. It doesn’t overwrite the need for sound scientific theory or validation of key variables, but it can provide a powerful evaluation (or repudiation) of hypotheses, and might even discover unexpected connections between variables that give rise to new theories for further testing.
Ethical
People struggle to be truly objective. We’re too susceptible to too many subconscious biases, even when we think we’re staying above the fray. The problem compounds when we design a machine learning model that uses historical data to make predictions—such data is tainted by previous decisions made by subconsciously biased humans, which now carries through to the model. “Blinding” the model doesn’t help: even when we withhold bias-laden information from a model (gender, ethnicity, etc.), an artificial intelligence designed to deduce underlying patterns in data can infer any bias from secondary variables in the historical data we feed it, and then parrot that bias right back to us in its predictions. The bias has then spread; the model is only as objective as its designer and its data. A tool like FACET, however, can smoke out this breed of bias, showing analysts how a model comes to its end predictions—and freely highlighting where it (like its human predecessors) based decisions off of inappropriate criteria. That’s profoundly impactful in processes that commonly struggle with implicit bias, like loan applications or immigration visas.
In short, Ittner and his team have expanded on a century of mathematical theory—and the wealth of the scientific AI community’s research—with a tool that could, in time, affect nearly every aspect of human life. If that sounds groundbreaking, well—that’s because it is.
“FACET is, in my knowledge, the first time you have a very robust tool—with a robust theory under it—to get that global overview of how the variables in the model are connected—what the model does,” Ittner said.
“That, for data scientists, is a huge leap.”
This story was produced by WIRED Brand Lab for Boston Consulting Group.
