Evaluating Explanation Methods by the Predictors They Induce
Jacob Selb{\ae}k, Hugo L. Hammer
cs.LG
Quick verdict
Turns explanations into parameter-free predictors to score PDP/ALE/SHAP/LIME; theory sound, controls strong; SHAP wins on dependent data, PDP if independent.
Summary
The paper proposes evaluating explanation methods by the predictors they induce: each explanation (PDP or ALE curves; SHAP or LIME attributions smoothed into per-feature curves by quantile binning) is assembled into an additive surrogate with unit coefficients — nothing is fitted except an intercept — and scored by out-of-sample R² against the black box's predictions. Under feature independence the authors prove, via a Hoeffding-decomposition argument, that this unit PDP surrogate is the optimal L2 additive summary of the model, so any OLS refitting gain becomes a diagnostic of feature dependence; experiments confirm gains within ±0.4% at ρ=0 and up to 59% (MLP) at ρ=0.85. Across 13 OpenML datasets, 9 synthetic designs and 4 model families, SHAP-derived curves reproduce black-box predictions best (R² 0.798 vs PDP 0.702, LIME 0.540), but the ranking is dependence-driven: under independence PDP is optimal as theory predicts, and an aggregator control shows the SHAP advantage lies in the attributions themselves. Turned on six explanation-quality metrics, a degradation ladder shows infidelity prefers a permuted explanation (0.347, below chance) and complexity/sparseness rank degraded explanations above intact ones, while the proposed metric matches faithfulness correlation (p=0.371). Limitations: no sensitivity sweep of the bin count, one dataset excluded post hoc for catastrophic PDP failure, and an inconsistent evaluation-cell count (25 vs 13+9).
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 1
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- An explanation's quality can be measured by the out-of-sample R² of the parameter-free additive predictor it induces against the black box (Eq. 1), with no fitting step so the score is attributable to the explanation alone.
- Under feature independence the unit-coefficient PDP surrogate is the optimal L2 additive summary of the model (Prop. 1), and the out-of-sample OLS refitting gain is a practical, distribution-free diagnostic of feature dependence (Cor. 2, validated in Table 3).
- On dependent real data SHAP-derived curves carry the most recoverable predictive content (R² 0.798 vs PDP 0.702, 12/13 datasets, p=0.005), but the ranking is entirely a dependence phenomenon: under independence PDP is provably optimal and SHAP does not beat it.
- Of six explanation-quality metrics tested on a degradation ladder of known ordering, infidelity prefers a permuted explanation (0.347) and complexity/sparseness systematically prefer degraded explanations, while the proposed metric matches faithfulness correlation (p=0.371).
Strengths
Parameter-free scoring with explicit credit attribution
The unit-coefficient surrogate (Eq. 3) fits only an intercept; Section 2.1.1 argues convincingly that a flexible aggregator would confound explanation content with learner capacity — the paper's sharpest departure from Pruthi et al.'s student-model precedent.
Correct, honestly positioned theory with useful corollaries
Prop. 1 (L2 additive projection under independence) is a Hoeffding-ANOVA identification the authors candidly relate to Molnar et al.'s IAS; the new content — Cor. 1 (refitting cannot help) and Cor. 2 (refitting gain as a dependence diagnostic) — is validated quantitatively in Table 3 (±0.4% at ρ=0; +59% MLP at ρ=0.85).
Unusually thorough negative controls
Disciplined statistics
Field-level payoff in the metric audit
Weaknesses
Arithmetic inconsistency in the cell count
Additive one-dimensional lens bounds all conclusions
No sensitivity analysis of conversion hyperparameters
Post-hoc exclusion of magic_telescope
Structural affinity in the metric-comparison test
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Preliminary Review: Evaluating Explanation Methods by the Predictors They Induce
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does it mean to evaluate an explanation by the predictor it induces?
The paper reads each feature's effect out of an explanation — a partial dependence (PDP) curve or a smoothed SHAP/LIME attribution — and adds the effects together into a simple additive predictor. How well that predictor reproduces the black-box model's predictions on unseen data (an R² score) measures how much predictive information the explanation carries. Nothing is fitted, so the score reflects the explanation itself rather than a surrogate model's capacity.
Which explanation method scored best?
On 13 real tabular datasets, SHAP-derived curves reproduced black-box predictions best (R² 0.798, versus 0.702 for PDP, 0.713 for ALE and 0.540 for LIME). However, the paper shows this ranking is caused by feature dependence: on synthetic data with independent features, PDP is provably optimal and SHAP does not beat it. Since real tabular features are almost always dependent, SHAP usually leads in practice.
Which explanation-quality metrics failed the paper's test?
Presenting six metrics with deliberately degraded explanations of known ordering, the paper found that infidelity preferred a permuted (worthless) explanation to the intact one (correct-ranking rate 0.347, below chance), and that complexity and sparseness systematically scored degraded explanations above faithful ones (means 0.324 and 0.346). Faithfulness correlation (0.980) and the paper's own predictive metric (0.967) tracked quality reliably.
Is code or data available for this paper?
Yes. The authors release all code, dataset specifications and full result tables at github.com/hugohammer/xaieval. The real datasets are public OpenML benchmarks with identifiers listed in the paper's Table 1, and the synthetic designs are fully specified by equations in Section 2.3.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Full text was extracted including all tables, but Figure 1's underlying data is only available through its caption and prose description.
- Mathematical notation is scrambled by text extraction in places (e.g., Eq. 16-17 formatting); all key equations remained legible.
- Review covers arXiv v1 (17 Sep 2026); the released code repository was not executed or inspected for this review.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.