arXiv paper2026-09-14· human-centered explainable AI (HCI / interpretable machine learning)· method paper (framework + case study + survey evaluation)

Data storytelling meets interpretable machine learning: Decoding AI decisions for non-experts without revealing sensitive data and model details

Q: What are the DIST Pyramid and the I-P-O model in this paper?

They are two conceptual artifacts proposed by the authors. The DIST Pyramid is a hierarchical model describing the progression from AI decision generation, through interpretation and data storytelling, to non-expert trust. The I-P-O model describes the interaction between an Input model (analytical/data models), a Process model (story models such as ABT, Freytag's Pyramid, SPSN), and an Output model (narrative presentation forms such as Martini Glass or interactive slide shows), with bidirectional feedback between stages.

Q: Did the ABT data stories actually outperform SHAP plots in the user study?

Yes, on most metrics. In the survey of 379 valid respondents, ABT stories beat SHAP plots on accessibility (76.1% vs 66.0% for why-not events; 74.3% vs 66.5% for what-if), contextualization, narrative coherence, and usefulness, with paired t-tests reaching significance on most comparisons. However, SHAP plots were rated better on model protection, privacy ratings were statistically indistinguishable, and the memory/emotional-resonance advantages were only significant for why-not events.

Q: How does the method protect sensitive data and model details?

Through two mechanisms: (1) differential privacy — a Gaussian noise mechanism with epsilon=5 applied to a designated sensitive feature before model training; and (2) explanation-granularity restriction — the generated stories expose only the user's input values, the prediction, and feature contributions, omitting model names, parameters, and code. The evaluation measured protection only through user perception questionnaires, not through actual privacy or information-leakage attacks.

Q: What dataset, model and LLMs were used in the case study?

The case study used the Boston Housing dataset (506 samples, 14 features, target = median house value), an XGBoost black-box regressor explained with SHAP, and three open LLMs to write the narrative text: Qwen3-235B-A22B (thinking and non-thinking modes), Llama-4-Scout-17B-16E-Instruct, and DeepSeek-R1-05/28, all at temperature 0 with chain-of-thought prompt templates.

Lemen Chao, Zixuan Yang, Anran Fang, Mingran Sun, Ming Lei

View original paper ↗arXiv:2609.15722

cs.AI · cs.CL · cs.HC · cs.LG

Quick verdict

SHAP-to-ABT storytelling framework for non-expert AI explanation; survey shows significant gains, but single-case evidence and small, uncorrected effects.

Summary

The paper proposes a framework that couples interpretable machine learning (IML) with data storytelling to explain black-box AI decisions to non-experts. Following design science research, the authors contribute a formal definition of storytelling-augmented IML, the DIST Pyramid, an I-P-O interaction model, and a five-module reference workflow (Predictor, Interpreter, Modeler, Scripter, Narrator) that converts model decisions into And-But-Therefore (ABT) narratives via what-if and why-not event generation, with privacy protection through data desensitization and restricted explanation granularity. A case study uses XGBoost on the Boston Housing dataset, SHAP attributions, and three LLMs (Qwen3-235B, Llama-4-Scout, DeepSeek-R1) to generate stories, which are mapped to Freytag, CAR, SPSN, and STAR structures. A survey (379 valid responses) compared ABT stories with SHAP plots: stories scored higher on accessibility (76.1%/74.3% vs ~66%), contextualization, narrative coherence, and usefulness (paired t-tests, mostly p<0.05), while SHAP plots were rated better on model protection and cognitive/emotional gains were inconsistent. Limitations include a single case study, modest effect sizes without multiple-comparison correction, unverified faithfulness of LLM-generated conclusions, and privacy evaluation based only on user perception rather than adversarial testing. No code or survey instruments are released.

Review complete

Assessment at a glance

human-centered explainable AI (HCI / interpretable machine learning)

risk flags identified

● high 0

● medium 4

● low 2

Reproducibility readinessmedium

Code & data availablenot stated

Review confidencemedium

Deep research viable

5 strengths · 5 weaknesses · 5 author questions

Levels reflect verification complexity — not a quality score. Evolve never rates your paper.

Key claims under review

Integrating data storytelling with IML yields explanations that non-experts rate significantly more accessible, contextualized, coherent and useful than SHAP visualizations (Tables 6-7).
The DIST Pyramid, I-P-O model, and five-module reference workflow constitute a general, automatable framework for converting black-box model decisions into structured narratives.
The ABT structure can serve as a canonical story form that is mechanically mappable to at least ten other storytelling frameworks (Table 3, demonstrated in Section 5.3).
Data desensitization during training plus restricted explanation granularity protects sensitive input data and model details while preserving explanation quality (Sections 5.1.2 and 5).

Strengths

Operational end-to-end pipeline, not just concepts
Section 4.4 maps the I-P-O abstraction onto a concrete five-module workflow (Predictor, Interpreter, Modeler, Scripter, Narrator), and Fig. 5 specifies what-if/why-not event generation as the mechanism for candidate story events — an implementable design rather than pure conceptual framing.
ABT as canonical form with systematized mapping
Table 3 aligns the ABT slots (And/But/Therefore) with ten other storytelling frameworks; Section 5.3 demonstrates the mapping on four concrete structures (Freytag's Pyramid, CAR, SPSN, STAR), which is a practical contribution for reusable narrative generation.
Quantitative user evaluation against a real baseline
Candid reporting of unfavorable results
Robustness check across LLM families

Weaknesses

Single toy case study; privacy is simulated, not tested
Statistical rigor gaps
Faithfulness of LLM narrative never validated
Why-not framed as causal inference
Survey methodology under-reported; no artifacts released

Full Review Report

$1.99 · one-time

Unlock the complete preliminary analysis Evolve produced for this paper.

Report preview

Scope & Claims

The paper targets the gap between technical IML outputs (SHAP, LIME, Anchors) and the comprehension needs of non-experts. Within a design science research paradigm (Section 3.1), it contributes four artifacts: a formal definition of storytelling-augmented IML (Eq. 4.1, composing desensitization D(x'), augmentation A(x'), contextualization C, and a structured storytelling function S), the DIST Pyramid, the I-P-O interaction model (Section 4.3), and a five-module reference workflow — Predictor, Interpreter, Modeler, Scripter, Narrator (Section 4.4). Two event-generation modes are defined: what-if (input perturbation, explicitly correlational) and why-not (output modification). The central claims: the pipeline automatically produces comprehensible ABT stories; ABT maps onto roughly ten other structures (Table 3); stories beat SHAP plots on user-facing metrics; and desensitization plus granularity limits protect sensitive data and model details.

Methodology Assessment

The case study (Section 5) is canonical but minimal: XGBoost on Boston Housing (506 samples, 14 features), SHAP attributions, and a Gaussian mechanism with epsilon=5 applied to the single 'tax' feature — a dataset the authors themselves admit contains no personal data (Section 5.1.2), so privacy is demonstrated rather than stress-tested. Three LLMs spanning reasoning and instruct modes generate the 'Therefore' text at temperature 0 from templated chain-of-thought prompts (Section 5.1.4, Table 4); And/But sections are strictly scripted 'to ensure objectivity' (Section

The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.

All 5 strengths & 5 weaknesses, analyzed in detail
6 risk flags with severity, section & reviewer questions
Methods & statistical evidence assessment
5 questions a peer reviewer would ask
Full six-dimension evaluation report & recommendations

One-time purchase · this paper only

One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.

Frequently asked questions

What are the DIST Pyramid and the I-P-O model in this paper?

They are two conceptual artifacts proposed by the authors. The DIST Pyramid is a hierarchical model describing the progression from AI decision generation, through interpretation and data storytelling, to non-expert trust. The I-P-O model describes the interaction between an Input model (analytical/data models), a Process model (story models such as ABT, Freytag's Pyramid, SPSN), and an Output model (narrative presentation forms such as Martini Glass or interactive slide shows), with bidirectional feedback between stages.

Did the ABT data stories actually outperform SHAP plots in the user study?

Yes, on most metrics. In the survey of 379 valid respondents, ABT stories beat SHAP plots on accessibility (76.1% vs 66.0% for why-not events; 74.3% vs 66.5% for what-if), contextualization, narrative coherence, and usefulness, with paired t-tests reaching significance on most comparisons. However, SHAP plots were rated better on model protection, privacy ratings were statistically indistinguishable, and the memory/emotional-resonance advantages were only significant for why-not events.

How does the method protect sensitive data and model details?

Through two mechanisms: (1) differential privacy — a Gaussian noise mechanism with epsilon=5 applied to a designated sensitive feature before model training; and (2) explanation-granularity restriction — the generated stories expose only the user's input values, the prediction, and feature contributions, omitting model names, parameters, and code. The evaluation measured protection only through user perception questionnaires, not through actual privacy or information-leakage attacks.

What dataset, model and LLMs were used in the case study?

The case study used the Boston Housing dataset (506 samples, 14 features, target = median house value), an XGBoost black-box regressor explained with SHAP, and three open LLMs to write the narrative text: Qwen3-235B-A22B (thinking and non-thinking modes), Llama-4-Scout-17B-16E-Instruct, and DeepSeek-R1-05/28, all at temperature 0 with chain-of-thought prompt templates.

About this review

This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.

The extracted text omits portions of Section 2, Section 4.2 (DIST Pyramid) and Section 6.1; the formal definition (Eq. 4.1) is garbled by PDF extraction, so the formal artifact was assessed from surrounding prose rather than the formula itself.
All figures (Fig. 4-10) are referenced but not analyzable from the text; quantitative judgments rely on the reported Tables 6-7.
Figures 6-9 are adapted from external blog sources per the authors' footnotes; the visual quality and consistency of the rendered data stories could not be assessed.
Survey demographics and recruitment details were not present in the extracted text; if they exist in a fuller version, the sampling-related criticism may partly soften.

Go deeper on this paper

Reproduce ResultsNot feasible for this paper — the core results cannot be re-derived by sandboxed code simulation.

Wrote a paper? Stress-test it before reviewers do.

Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.

Scan your paper