Faithful Faithfulness Evaluations: Challenges & Pitfalls Learned from a Breast MRI Case Study
Peachapong Poolpol, Henrik H. J. Detjen, Eike Petersen
cs.CV · cs.HC · cs.LG · eess.IV
Quick verdict
Breast-MRI ViT case study showing saliency faithfulness rankings hinge on perturbation strategy and class-specificity; transparent, code-backed, but single-task.
Summary
This protocol-sensitivity study asks whether perturbation-based faithfulness evaluations of saliency maps yield stable conclusions. The authors evaluate seven saliency methods (Last-layer Attention, Attention Rollout, Grad-SAM, Gradient Attention Rollout, GMAR, Grad-CAM, HiResCAM) on a Medical Slice Transformer with DINOv3-ViT backbone trained on the public ODELIA 2025 breast MRI dataset (three classes; 792/114/116 patient-level split). Deletion/insertion AUCC was computed under four perturbation strategies. Under fixed-intensity masking (black, white, minimum-intensity) virtually all real explanations scored close to random perturbation — e.g., deletion AUCC 0.346-0.358 vs 0.355 random under black masking — confirming that out-of-distribution artifacts confound rankings. Attention masking produced far wider separation (0.336-0.512), with Grad-CAM best on full deletion (0.336) and insertion (0.637), and Gradient Attention Rollout best on early deletion (0.110). To compare fairly with class-agnostic attention methods, they introduce class-averaged variants of gradient methods; class-specific advantages largely vanish, and class-averaged Grad-CAM falls to random-level deletion (0.475), indicating the advantage stems mainly from access to class information. Key limitations: single dataset, single architecture, and only moderate model performance, so rankings may not generalize. Code is public.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 4 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Perturbation-based faithfulness rankings of saliency methods depend strongly on the perturbation strategy: under fixed-intensity replacements (black/white/min-intensity) real methods score close to the random baseline, while attention masking yields clear separation.
- Attention masking is the most principled removal strategy for ViT classifiers and should be preferred in medical-imaging faithfulness studies.
- Under attention masking, Grad-CAM and Gradient Attention Rollout are the strongest class-specific methods, with Grad-CAM best on full deletion (0.336) and insertion (0.637) and Gradient Attention Rollout best on early deletion (Deletion30 0.110).
- Fair comparison across explanation families requires class-agnostic variants: the class-averaged variants introduced here show that much of the class-specific methods' advantage comes from access to class information rather than the explanation mechanism itself.
- This is the first comprehensive faithfulness evaluation of seven saliency methods in a ViT-based volumetric (3D) medical image classification setting.
Strengths
Direct quantitative demonstration that perturbation choice flips rankings
Table 1 shows that under fixed-intensity replacements all seven class-specific methods score within noise of the random baseline (e.g., black masking: methods 0.346-0.358 vs random 0.355; min-intensity: random 0.476 vs Grad-CAM 0.323 vs Grad. Roll. 0.327), whereas attention masking separates methods clearly (0.336-0.512). This converts a methodological caution into actionable evidence.
Principled protocol choice aligned with ViT-specific literature
Attention masking removes patches inside the transformer rather than injecting out-of-distribution values (Sec. 3.4), following recommendations the authors note are rarely applied in medical imaging (Sec. 2.2, refs 12, 13).
Class-averaged non-class-specific variants make cross-family comparison fair
Honest, caveat-aware reporting
Reproducibility posture
Weaknesses
No statistical testing behind ranking claims; overlapping CIs throughout
Single dataset, single architecture, moderate-performing model
Insertion baseline underspecified; no retraining-based ground truth
Interpretation questions the text leaves open
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a protocol-sensitivity case study, not a new explanation method. Using a Medical Slice Transformer (DINOv3-ViT backbone, Sec. 3.1) trained on the public ODELIA 2025 breast MRI dataset (1,022 unilateral studies, three classes; patient-level split 792/114/116), the authors perturbation-test seven saliency methods — Last-layer Attention, Attention Rollout, Grad-SAM, Gradient Attention Rollout, GMAR (L1/L2), Grad-CAM, HiResCAM — plus non-class-specific variants (Sec. 3.2). Central claims: (1) method rankings depend on the perturbation strategy; (2) fair benchmarking must separate class-specific from class-agnostic explanations; (3) Grad-CAM and Gradient Attention Rollout are the strongest class-specific methods, with ranking between them design-dependent. The evaluation itself uses only perturbation AUCC — no lesion-localization ground truth — so 'faithfulness' here means agreement with model behavior, not clinical correctness (a distinction the authors make explicitly in Sec. 5).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does the paper 'Faithful Faithfulness Evaluations' show?
It shows, through a breast MRI classification case study, that rankings of saliency-map explanation methods in perturbation-based faithfulness tests depend heavily on the evaluation protocol: which replacement/masking strategy is used and whether explanations are class-specific or class-agnostic. Fixed-intensity masking made almost all methods perform like random perturbation, while attention masking separated methods clearly.
Which saliency methods were compared and which performed best?
Seven methods: Last-layer Attention, Attention Rollout, Grad-SAM, Gradient Attention Rollout, GMAR, Grad-CAM, and HiResCAM. Under attention masking, Grad-CAM achieved the best full deletion (0.336) and insertion (0.637) AUCC among class-specific methods, and Gradient Attention Rollout was best on the early-deletion metric (0.110); no method dominated all settings.
Why is fixed-value masking (e.g., black patches) a problem in faithfulness evaluations?
Replacing image patches with extreme out-of-distribution values creates unnatural inputs, so confidence changes reflect artifacts rather than removed evidence. In this study under black masking, all real explanation methods scored essentially like random perturbation (deletion AUCC about 0.346-0.358 vs 0.355 random), making rankings meaningless.
How well did the underlying breast MRI model perform?
Moderately: accuracy 0.57, macro F1 0.52, macro AUROC 0.73 on the three-class task (malignant one-vs-rest AUROC 0.81, precision 0.77, recall 0.53), reflecting how difficult the ODELIA breast MRI classification task is.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 (saliency examples and confidence curves) could not be analyzed from the extracted text; qualitative claims in Sec. 4.2 are taken from the prose only.
- Implementation details not present in the text (patch-to-slice mapping, exact masking mechanics, training hyperparameters, exact split lists) could not be verified; the public code repository was not inspected.
- AUCC normalization and the insertion baseline description are ambiguous in the text, so interpretation of absolute AUCC magnitudes is provisional.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.