Explainability Assistant: A Conversational XAI Interface for Interpreting Energy Consumption Models
Rodion Krjutškov, Eduard Barbu, Nikos Sakkas, Sofia Yfanti
cs.AI · cs.LG
Quick verdict
Claims an LLM function-calling XAI chatbot hits 94% intent-parsing vs 76.8% prior; system is open and reproducible, but expert evidence rests on just 3 users.
Summary
The paper introduces the Explainability Assistant (EA), an open-source conversational XAI system that replaces custom grammars with LLM function-calling so users can query ML model predictions in natural language. A Next.js front-end and FastAPI back-end orchestrate SHAP attributions, DiCE counterfactuals, and what-if simulations; the LLM sees only queries and tool schemas, while data and models stay local. Two use cases are shown: office-building energy regression (a genetic-programming symbolic regressor) and heart-disease classification, the latter migrated by JSON configuration alone without fine-tuning. On two 100-sample gold parse datasets (A built manually; B and C generated by GPT-5 and Gemini-2.5-Pro, then manually verified), Gemini-2.5-Flash achieved 94% (A+B) and 93% (A+C) exact-match intent-parsing accuracy, above the 74-91% of GPT-5-mini, Llama-3.3-70B, and Gemini-2.0-Flash, and above the 76.8% reported for the grammar-based TalkToModel. A within-subjects study with three energy experts found 100% task accuracy versus 93% for a traditional dashboard; all three rated ease of use and future-use intent at maximum, while trust and system-understanding ratings stayed moderate (3-5). Main limitations: a tiny expert sample, fixed interface order, evaluation data generated by LLMs from the same families as evaluated models, and an uncontrolled cross-paper baseline comparison.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LLM function-calling achieves up to 94% exact-match intent-parsing accuracy (Gemini-2.5-Flash on gold set A+B), versus the 76.8% reported for grammar-based TalkToModel.
- The system adapts to new ML problem types — energy regression to heart-disease classification — via JSON function-specification changes alone, with no task-specific fine-tuning.
- In a three-expert within-subjects study, the conversational interface matched or exceeded a traditional dashboard on task accuracy (100% vs 93%), with all experts rating ease of use and future adoption at maximum.
- Executing all functions in the back-end lets proprietary datasets and model internals remain local, with only queries and tool schemas sent to LLM providers.
Strengths
Complete open artifact stack
Appendix B links the front-end and back-end repos, gold parse datasets with generation prompts, evaluation scripts, a live demo, a video, and the full expert questionnaire — rare completeness for a system paper and the basis for high reproducibility.
Systematic multi-model parsing evaluation
Section 4.1 evaluates four LLMs on two combined 100-sample gold sets (A manual + B/C LLM-generated and manually verified) with exact-match scoring; Table 2 reports Gemini-2.5-Flash at 94%/93%, Gemini-2.0-Flash at 88%/91%, GPT-5-mini at 74%/79%, Llama-3.3-70B at 78%/74%.
Privacy- and transparency-aware architecture
Candid limitations appendix
Demonstrated task portability
Weaknesses
Human evidence rests on three experts with a confounded order
Generation/evaluation family overlap in the parsing benchmarks
Cross-paper baseline comparison is not controlled
No variance or robustness quantification
Favorable reading of moderate trust ratings
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper presents the Explainability Assistant (EA), an open-source conversational XAI system that replaces the custom grammar of TalkToModel with native LLM function-calling. Positioning (Section 2) is clear: TalkToModel fine-tunes T5 models against task grammars (76.8% parsing accuracy); LLMCheckup relies on prompt chaining; Samimi et al. retain a fine-tuned T5 parser beside LLM dialogue. EA claims three contributions (Section 1): an LLM function-calling architecture for XAI dialogue, a modular and easily redeployed design, and empirical evidence of improved model understanding for domain experts. Scope is prediction-level explanation for end users — SHAP attributions, DiCE counterfactuals, what-if simulation — not mechanistic interpretability in the style of MAIA. The headline claims are quantitative (94% intent-parsing in Section 4.1, Table 2; unanimous expert preference in Section 4.3), so this review asks whether the evidence supports exactly those numbers.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the Explainability Assistant?
An open-source chatbot that lets non-experts ask natural-language questions about ML model predictions. It uses LLM function calling to invoke SHAP feature-importance analysis, DiCE counterfactuals, and what-if simulations on a local backend, demonstrated on building energy forecasting and heart-disease classification.
How much better is it than TalkToModel?
The paper reports 94% intent-parsing accuracy with Gemini-2.5-Flash versus TalkToModel's reported 76.8%, but the two systems were measured on different datasets, grammars, and function sets, so this is an indicative cross-paper comparison rather than a controlled benchmark.
How many experts participated in the user study?
Three energy-domain professionals compared the conversational interface with a traditional XAI dashboard in a within-subjects study. Task accuracy was 100% vs 93% and all three preferred the chat interface for usability, but a sample of three is far too small for statistical significance.
Is the code and evaluation data available?
Yes. The paper links front-end and back-end repositories, the gold parse datasets with their generation prompts, LLM evaluation scripts, a live demo, a video walkthrough, and the full expert questionnaire (Appendix B).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 are image content; only their captions and the surrounding prose were extracted, so the plotted detail of Figure 3 (individual Likert trajectories) was assessed from the text's description, not the plot itself.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.