SHAP: From Game Theory Tables to Model Explainability

In 1953, Lloyd Shapley, a mathematician at RAND Corporation, published a paper answering a deceptively simple question: if a group of players cooperate to win a game, how do you fairly divide the reward? His answer was an elegant formula that computed each player's "marginal contribution" averaged over all possible orderings of who joined the coalition. He called these the Shapley values.

The paper was purely theoretical. Shapley was interested in cooperative game theory, abstract problems about coalitions, fairness, and incentives. He could not have imagined that 65 years later, his formula would be applied to explain why a machine learning model decided to deny someone a loan.

The leap happened gradually. Economists used Shapley values to split costs fairly between cities sharing infrastructure. Political scientists used them to measure the voting power of individual states in weighted voting systems. But the connection to machine learning lay dormant until 2017, when Scott Lundberg and Su-In Lee published "A Unified Approach to Interpreting Model Predictions," introducing SHAP.

The insight was elegant: treat a machine learning model as a cooperative game. Each feature in your dataset is a "player." The model's prediction is the "prize." Shapley values then tell you exactly how much each feature contributed to any individual prediction, consistently, fairly, and with mathematical guarantees about additive accuracy, missingness, and consistency that no prior method had satisfied simultaneously.

Today, SHAP is one of the most widely used model explainability libraries in data science. Every time a bank explains a credit decision, a doctor queries why an AI flagged a scan, or a regulator audits an algorithmic system, there's a good chance Shapley's 1953 poker-table math is doing the heavy lifting. Fair division, it turns out, is a universal problem.