SHAP values are a method for explaining individual predictions made by machine learning models, assigning each input feature a number that represents how much it pushed the prediction up or down. The name stands for SHapley Additive exPlanations, a framework introduced by Scott Lundberg and Su-In Lee that draws on a decades-old concept from cooperative game theory and applies it to the problem of understanding why a model made a particular decision.1arXiv. A Unified Approach to Interpreting Model Predictions The approach has become one of the most widely used tools in explainable AI, but it carries assumptions and limitations that matter in practice.
What SHAP Values Actually Tell You
Imagine a model predicts that a loan applicant has a 78% chance of defaulting. That number alone is not very useful to the person reviewing the decision. SHAP values break it apart: maybe the applicant’s high debt-to-income ratio pushed the prediction up by 15 percentage points, their long employment history pulled it down by 8 points, their recent missed payment added another 12 points, and so on. Every feature gets a signed value. Positive means it raised the prediction; negative means it lowered it. When you add all the SHAP values together with the model’s average prediction across all applicants, you land on the 78% figure.
This additive property is not just a convenience. SHAP’s foundational paper showed that it is the unique method satisfying a set of desirable mathematical properties borrowed from game theory, including the requirement that the feature contributions always sum exactly to the difference between the model’s prediction and its baseline average.2arXiv. A Unified Approach to Interpreting Model Predictions That guarantee is what distinguishes SHAP from ad hoc importance scores that many models produce internally. A random forest, for instance, can tell you which features it split on most often, but that does not tell you how much each feature contributed to any single prediction.
The Axiomatic Foundation
The reason SHAP values have the properties they do traces back to Lloyd Shapley’s 1953 result in game theory. Shapley showed that if you want to fairly divide a payout among players in a cooperative game, there is exactly one way to do it that satisfies a small set of fairness rules. SHAP treats each feature as a “player” and the model’s prediction as the “payout,” then applies that same logic.
The original SHAP paper claimed that a handful of axioms were sufficient to single out Shapley values as the only solution. More recent mathematical work has refined that claim. A rigorous analysis showed that the symmetry axiom, which says two features that contribute identically should receive the same value, is actually essential to the proof, contradicting an earlier claim that it was redundant.3arXiv. Mathematically rigorous proofs for Shapley explanations For practitioners this is mostly a theoretical detail, but it matters when people try to propose alternative explanation methods that drop one of the axioms: the uniqueness guarantee breaks down, and you may end up with explanations that behave inconsistently.
From Individual Predictions to Global Understanding
SHAP values are computed per prediction, but they become even more powerful when you compute them for many predictions at once and look at patterns across the whole dataset. If you run SHAP on every sample, you get a matrix with one row per data point and one column per feature, each cell holding a SHAP value.4Interpretable Machine Learning. SHAP From that matrix, several standard visualizations emerge.
The simplest is a bar chart of global feature importance. You take the average of the absolute SHAP values for each feature across all data points. Features with large average absolute values are, on the whole, more influential to the model’s output. This is analogous to other feature importance rankings, but because it is derived from Shapley values, it has the additive consistency that simpler importance measures lack.
More informative is the summary plot, sometimes called a beeswarm plot. Each dot represents one SHAP value for one data point. The feature sits on the vertical axis, the SHAP value on the horizontal axis, and the dot’s color indicates whether the feature’s actual value was high or low. So you can see at a glance that, say, high values of a feature consistently push the prediction up (dots are red and to the right), while low values pull it down (dots are blue and to the left). When the pattern is not that clean, the scatter of dots tells you the relationship is more complex or depends on interactions with other features.5Interpretable Machine Learning. SHAP
Dependence plots go one step further. You pick a single feature, plot its raw value on the x-axis and its SHAP value on the y-axis, and color by a second feature to surface interactions. If the relationship between the feature and its SHAP value shifts depending on the value of the second feature, that is evidence of an interaction effect the model has learned.
How SHAP Compares to LIME
LIME (Local Interpretable Model-agnostic Explanations) is the other widely used post-hoc explanation method, and the two get compared constantly. The key practical difference is scope. LIME generates an explanation for a single prediction by fitting a simple surrogate model in the neighborhood of that data point, so its explanations are inherently local. SHAP can do the same local explanation, but because the values are additive and consistent, you can also aggregate them across many predictions to get global insights about the model.6Scientific Reports. Machine learning explainability in nasopharyngeal cancer survival using LIME and SHAP
LIME is often faster to compute for a single explanation, and its surrogate model approach is easy to grasp intuitively. But LIME’s explanations can be unstable: run it twice on the same prediction with different random seeds and you may get different feature rankings. SHAP values, because they are grounded in game theory, are deterministic given the same model and data. That stability is a big reason SHAP has become the default in regulated industries where you need to justify a decision repeatedly.
Algorithms Under the Hood
Computing exact Shapley values is expensive. For a model with, say, 30 features, you would theoretically need to evaluate the model on every possible subset of features, and the number of subsets grows exponentially. In practice, nobody does this. Instead, different algorithms approximate or exploit model structure to make things tractable.
For tree-based models like gradient-boosted trees and random forests, TreeSHAP is the standard. It takes advantage of the tree structure to compute exact SHAP values in polynomial time rather than exponential time.7arXiv. Consistent Individualized Feature Attribution for Tree Ensembles An optimized version, Fast TreeSHAP, pushed performance further, with benchmarks showing it runs roughly 1.5 to 2.5 times faster than the original TreeSHAP implementation depending on the variant used.8arXiv. Fast TreeSHAP: Accelerating SHAP Value Computation for Trees
For deep neural networks, DeepSHAP combines the Shapley framework with backpropagation-style attribution, propagating contributions through the network’s layers. Researchers have extended this to specialized architectures, including complex-valued neural networks used in signal processing and radar applications.9arXiv. DeepCSHAP: Utilizing Shapley Values to Explain Deep Complex-Valued Neural Networks For models where no specialized algorithm exists, KernelSHAP offers a model-agnostic approach that works with any black-box model by using weighted sampling to approximate the Shapley values.
The choice of algorithm matters for speed and, in some cases, accuracy. TreeSHAP gives you exact values for tree models. DeepSHAP involves approximations that trade some theoretical purity for speed on large networks. KernelSHAP is the most flexible but the slowest, and its approximation quality depends on the number of samples used. For a model deployed in production where explanations need to be generated on the fly, these tradeoffs are real engineering decisions.
The Correlated Features Problem
SHAP’s theoretical elegance rests on one assumption that rarely holds in practice: feature independence. The standard approach to computing Shapley values involves asking “what would the model predict if we removed feature X?” Because you cannot literally delete an input from a trained model, the method replaces the missing feature with values drawn from the data. If it draws those replacement values independently of the other features, and the features are actually correlated, the result is data points that could never occur in reality.
Consider a model that uses both height and weight as inputs. If you try to evaluate the contribution of height by replacing it with random values while holding weight fixed, you may create combinations like a person who is 5 feet tall and weighs 250 pounds, or 6 foot 5 and weighs 90 pounds. The model was never trained on such inputs, and its behavior on them is unpredictable. When features are correlated, this process can produce misleading explanations.10Artificial Intelligence. Explaining individual predictions when features are dependent: More accurate approximations to Shapley values
Researchers have proposed alternatives that condition on the observed relationships between features rather than treating them as independent. These “conditional” SHAP approaches avoid creating impossible data points, but they introduce a different problem, which connects to a deeper question about what SHAP explanations actually mean causally.
Marginal Versus Conditional, and Why It Matters Causally
The correlated-features debate leads directly into one of the more active theoretical discussions around SHAP: should the method use marginal or conditional expectations when features are missing? The marginal approach replaces absent features using their overall distribution, ignoring the values of the features that are present. The conditional approach replaces them using values consistent with the observed features. Both sound reasonable, and they can give very different answers.
The conditional approach has an intuitive appeal: it avoids those unrealistic data points. But from a causal perspective, it has a fundamental flaw. It cannot distinguish between a feature that genuinely causes the outcome and a feature that is merely correlated with a causal feature. If two features are highly correlated because one causes the other, the conditional approach will spread credit between them, even if only one actually drives the model’s prediction.11arXiv. Causal Analysis of Shapley Values: Conditional vs. Marginal
Work on causal Shapley values has shown that the standard observational conditional distribution is unable to separate genuine causal effects from spurious correlations. An interventional approach, which conceptually asks “what would happen if we set this feature to a particular value, breaking its correlations with everything else,” would be more appropriate for causal questions. But interventional SHAP requires knowledge of the causal structure among features, which practitioners rarely have.12arXiv. Revisiting Shapley Values: A Causal Perspective In practice, most users stick with the marginal approach and accept that the explanations describe associations the model has learned, not necessarily causal relationships in the world.
Adversarial Attacks That Fool SHAP
A more troubling limitation is that SHAP explanations can be deliberately manipulated. Researchers have demonstrated multiple attack strategies that allow a malicious actor to build a model that behaves in a biased way but produces SHAP explanations that look completely fair.
One approach, called scaffolding, wraps a biased classifier inside an outer model that detects when SHAP is probing it and returns innocuous-looking responses. The biased predictions remain unchanged on real inputs, but the post-hoc explanations are crafted to hide the bias. In experiments using real-world datasets including the COMPAS recidivism dataset, researchers showed that extremely biased classifiers could fool both LIME and SHAP into generating explanations that appeared unbiased.13arXiv. Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
Later work relaxed the requirements for carrying out such attacks. “Shuffling attacks” showed that an adversary does not even need access to the underlying data distribution to fool SHAP; the attack can be carried out in a data-agnostic way by adapting any trained model to produce misleading Shapley value explanations.14arXiv. Fooling SHAP with Output Shuffling Attacks A separate line of work demonstrated that you do not even need to modify the model itself. By subtly biasing the background data samples that SHAP uses to compute expectations, an attacker can manipulate explanations while leaving the model completely untouched.15arXiv. Fool SHAP with Stealthily Biased Sampling
These findings do not mean SHAP is useless, but they put an important boundary on what it can guarantee. SHAP tells you how a model distributes credit among features. It does not tell you whether the model is trustworthy, or whether the explanations themselves have been tampered with. In high-stakes settings like criminal justice or lending, SHAP explanations should be treated as one piece of evidence within a broader audit, not as proof that a model is fair.
SHAP in Fairness Auditing
Despite those vulnerabilities, SHAP has become a go-to tool for investigating bias in machine learning models, particularly in finance. A study of credit scoring models used SHAP to show how variables like marital status, age, and credit limit can act as proxies for gender, allowing a model to maintain discriminatory patterns while appearing on the surface to satisfy fairness constraints.16arXiv. Structural Gender Bias in Credit Scoring: Proxy Leakage The model never sees gender directly, but SHAP reveals that features closely correlated with gender carry large attribution values, which is a red flag for regulators.
This kind of analysis is why SHAP has become so popular in regulated industries. When a bank denies a loan, regulators increasingly want to see which features drove the decision. A SHAP explanation can point to specific features and quantify their contribution. That does not replace domain expertise in deciding whether the features are legitimate, but it makes the model’s reasoning visible enough to have that conversation.
Beyond Tabular Data
SHAP started in the world of tabular data, where features are clean columns like age, income, and blood pressure. But the framework has expanded. For image models, SHAP values can be computed over superpixels or image patches rather than individual pixels. For text models, the “features” might be tokens or sentences.
One interesting extension applies SHAP to multimodal models that combine different types of input, like vision-and-language systems. MM-SHAP uses Shapley values to measure how much each modality (image versus text) actually contributes to the model’s predictions. This can reveal, for example, that a visual question-answering model is mostly relying on the text and barely looking at the image, despite being designed to use both.17ACL Anthology. MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks For researchers building these systems, that kind of diagnostic is invaluable: it tells you whether your model has actually learned to integrate its inputs or is taking shortcuts.
How Different Explanation Methods Compare in Practice
SHAP is not the only feature attribution method, and choosing between options is not straightforward. A benchmarking study that compared DeepSHAP against several other attribution techniques for neural networks, including permutation-based methods and sensitivity indices, found that the explanations produced by different methods often disagree with each other. How well each method’s output matched human intuition depended on the properties of the method itself.18ACM Computing Surveys. A review and benchmark of feature importance methods for neural networks
This is worth internalizing: there is no ground truth for feature importance in most real-world problems. When SHAP says feature A is the most important and another method says it is feature B, there is no oracle to consult. The theoretical guarantees behind SHAP (additivity, consistency, symmetry) give it a principled foundation, but those guarantees are about internal coherence, not about whether the explanation matches reality. A model that has learned a spurious correlation will produce SHAP values that faithfully describe that spurious correlation. SHAP explains the model, not the world.
In practice, many teams use SHAP alongside other tools rather than in isolation. Agreement across methods increases confidence; disagreement signals that the model’s reasoning deserves a closer look. That layered approach is especially important given the adversarial vulnerabilities discussed above. No single explanation method is a complete answer to the question “why did the model do that?” but SHAP, with its mathematical grounding and flexible visualization toolkit, remains one of the most useful starting points available.