My stakeholders are refusing to deploy my Random Forest model because they call it a "black box." They want to know exactly why a specific loan application was rejected. Is SHAP (SHapley Additive exPlanations) the industry standard for this, or are there simpler methods that are easier to explain to non-technical managers?
3 answers
SHAP is definitely the gold standard because it is based on solid game theory and ensures that the "contribution" of each feature adds up to the total prediction. For stakeholders, I recommend using "Summary Plots" for global importance and "Force Plots" for individual decisions. If SHAP is too computationally expensive for your dataset, you could try LIME (Local Interpretable Model-agnostic Explanations), which builds a simple linear model around a specific prediction to explain it. However, SHAP is more consistent. It’s been a game-changer for getting buy-in from risk and compliance teams in the banking sector.
Do you find that SHAP values are stable when you have highly correlated features, or do the explanations start to become misleading?
If they want something even simpler, try Partial Dependence Plots (PDPs). They show the marginal effect one or two features have on the predicted outcome.
PDPs are great for showing trends, Valerie. They are much easier for a manager to digest than a complex SHAP waterfall chart at first glance.
Trevor, that is a known issue. When features are correlated, SHAP can split the "credit" between them, which might hide the true importance of a variable. To fix this, I usually perform feature selection to remove redundancy before running SHAP. This ensures that the explanation provided to the stakeholders is both clear and statistically sound, avoiding any confusion during the audit process.