XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?
Yanfei Hu Fleischhauer, Alona Zharova, Nadja Klein, Stefan Feuerriegel
cs.AI
Quick verdict
LLM-as-judge framework (GPT-5.4) benchmarking 5 XAI methods on 8 quality dimensions; solid design, but human validation thin and stats pseudoreplicated.
Summary
The paper introduces XAI-Arena, an LLM-as-a-judge framework in which a single LLM (GPT-5.4, temperature=0) rates XAI explanation quality on eight Likert-scored dimensions — perceived simplicity, clarity, task adequacy, trust calibration, actionability, transparency, faithfulness, interpretability — under four stakeholder personas (ML developer, data scientist, manager, end user). It is instantiated over five XAI methods (SHAP, LIME, DiCE, PDP, permutation importance), four models (logistic regression, random forest, XGBoost, MLP), and synthetic plus three real-world tabular datasets, producing 17,252 evaluation records (138,016 dimension ratings). Key results: method effects are large (omega2 up to .869 for faithfulness), with SHAP leading on faithfulness/transparency, DiCE on actionability, permutation importance on interpretability; persona effects are very large (omega2 .711 for perceived simplicity), with the End User persona lowest; model effects are smaller and concentrate on transparency. Human validation with 48 Prolific participants on one Telco Churn case found Spearman rho = .693 between aggregate human and LLM ratings over 32 artifact-dimension means. Proxy-metric alignment is mixed: stability metrics correlate strongly with LLM-rated faithfulness (rho .66-.75) while MoRF AUC does not (rho = -.127). Main limitations: the human study covers a single case and four text-only artifacts (the actionability correlation is strongly negative, -.894), ratings are clustered by artifact yet analyzed as independent, cross-LLM correlations are sometimes near zero, and no repeated-call variance is assessed.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LLMs can serve as a reproducible, scalable, multidimensional, and stakeholder-sensitive mechanism for comparative assessment of XAI explanation quality (the XAI-Arena framework).
- Perceived explanation quality differs systematically across XAI methods, and no method dominates all eight dimensions: SHAP leads faithfulness/transparency, DiCE actionability, permutation importance interpretability (Sec. 4.4, E.8).
- Stakeholder persona strongly conditions ratings, with the End User persona assigning the lowest scores (omega2=.711 for perceived simplicity, Table 8).
- LLM ratings align with human ratings at the aggregate level (Spearman rho=.693, p<.001, over 32 artifact-dimension means, Sec. 4.6.2).
- XAI-Arena is positioned as a complementary evaluation layer to human studies and proxy metrics, not a replacement (Sec. 5.2).
Strengths
Unusually systematic factorial design
Five XAI methods x four models x synthetic (4 sample sizes x 3 feature counts x linear/nonlinear) + 3 real tabular datasets x 4 personas x 8 dimensions, yielding 17,252 records / 138,016 ratings (Sec. 3.3, 4.1), enabling clean method/persona/model contrasts no prior human-study literature covers at this scale.
Above-average reproducibility hygiene
Full prompt template in Appendix B, fixed decoding (temperature=0, top_p=1.0), fixed seeds, version-controlled pipeline, dataset/model/artifact validation notebooks (Appendices C-E), GUIDE-LLM checklist, and a public anonymous code+data repo (footnote 1).
Modern statistical reporting with effect sizes
Honest framing of scope
Multi-channel external validation attempt
Weaknesses
Pseudoreplication inflates all inferential statistics
Headline human-LLM agreement rests on a very thin base
Cross-LLM robustness claim overstates Table 10
Faithfulness dimension lacks any ground truth
Persona effects may be LLM stereotyping, not stakeholder signal
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
XAI-Arena — Structured Preliminary Review
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is XAI-Arena?
XAI-Arena is a framework from this 2026 arXiv paper that uses an LLM (GPT-5.4) as an automated judge to score the quality of explainable-AI (XAI) explanations — produced by methods like SHAP, LIME, DiCE, PDP and permutation importance — on eight dimensions (e.g., clarity, faithfulness, actionability) from four stakeholder perspectives, enabling large-scale, reproducible benchmarking of XAI methods on tabular datasets.
How well do LLM ratings match human judgments of XAI explanations?
In the paper's human study, 48 online participants rated explanations for a single Telco churn case. Aggregating to 32 artifact-dimension means, human and LLM ratings correlated at Spearman rho=.693. However, alignment varied sharply by dimension — from rho=1.000 (transparency, simplicity) down to rho=-.894 (actionability) — so the agreement claim holds at the aggregate level, not per dimension.
Which XAI explanation method did the LLM judge as best?
No single method dominated all eight dimensions. SHAP scored highest on faithfulness and transparency, DiCE counterfactual explanations on actionability, and permutation importance on perceived simplicity, trust calibration and interpretability. The authors conclude that XAI method choice should follow the explanation goal and intended stakeholder rather than a single overall ranking.
What are the main limitations of the XAI-Arena paper?
Key limitations include reliance on a single primary LLM (GPT-5.4) with mixed cross-model correlations (some near zero in Appendix H), a narrow human validation (one dataset, one case, four text-only artifacts), tabular data only, ratings clustered by artifact that are analyzed as independent in the statistics, no measurement of run-to-run LLM variability, and the fact that the LLM cannot truly verify the 'faithfulness' of an explanation to the model's internal reasoning.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure panels (Fig. 3, 9, 10 heatmaps) extract as raw number streams; all statistics cited in this review come from the running text and numbered tables, not decoded figures.
- Supplementary materials referenced in the text (GUIDE-LLM checklist, full survey materials) are external and not part of the provided extraction.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.