An Empirical Study of Counterfactual Self-Explanations in LLMs
Giannis Kalyvas, Giorgos Filandrianos, Orfeas Menis Mastromichalakis, Vassilis Lyberatos, Giorgos Stamou
cs.CL
Quick verdict
Claims LLM counterfactual self-explanations improve with model scale (ρ=0.87 flip rate); credible trends, but aggregate-level statistics temper the causal claim.
Summary
The paper asks whether LLMs can produce faithful counterfactual self-explanations: minimal edits to an input that flip the model's own prediction. Across sentiment analysis (ERASER movie reviews) and natural language inference (e-SNLI), the authors evaluate ten instruction-tuned models (LLaMA-3: 1B-70B; Qwen-2.5: 1.5B-72B) under three prompting conditions (baseline, chat-history, rationale-guided), scoring flip rate, closeness (edit distance plus MPNet semantic similarity), and a new metric ESMP measuring whether edits land inside human-annotated evidence spans. Larger models flip their own predictions far more often (Spearman rho=0.87; +11.9 percentage points per parameter doubling in a controlled regression; fractional-logit odds ratio 2.00) and target human-aligned evidence more (rho=0.78), while closeness and similarity are nearly flat. Chat-history prompting maximizes faithfulness (best in 12/20 settings); rationale-guided prompting maximizes ESMP and closeness (15/20 each) but lowers faithfulness, showing plausibility and faithfulness can dissociate. Main limitations: binary tasks only, two model families, setting-mean-level statistics, no instance counts reported, and scale trends partly confounded by distillation within the LLaMA family.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Model scale is the strongest determinant of counterfactual self-explanation faithfulness: flip rate correlates with size at rho=0.87, and each parameter doubling adds about 11.9 percentage points of faithfulness under controlled regression.
- Larger models also produce edits concentrated on human-annotated evidence, with size-ESMP correlation rho=0.78 and +3.2 points of ESMP per size doubling.
- Rationale-guided prompting yields more edit-minimal and more human-aligned counterfactuals (best in 15/20 settings each) but does not consistently improve, and can reduce, faithfulness.
- Chat-history prompting produces the most faithful self-explanations, winning in 12 of 20 settings.
- Counterfactual self-explanations are useful behavioral evidence but should be empirically validated rather than assumed, since their reliability depends strongly on model capacity.
Strengths
Controlled multi-family scale design
Ten instruction-tuned models across two families (Tables 2-5) with within-family comparisons, plus a regression on log2(size) controlling for dataset, family, and prompting; three analyses converge (Spearman rho=0.866, beta=0.119 per doubling with CI [0.092, 0.147], fractional-logit OR 2.00 [1.68, 2.40]).
ESMP is a well-motivated new metric
Section 2.3 defines Evidence-Supported Modification Precision as the precision of edits landing inside ERASER-annotated evidence spans, and explicitly justifies precision over recall to avoid rewarding non-minimal edits - a design choice consistent with the minimality objective.
Actionable dissociation finding
High transparency of setup
Honest scope statements
Weaknesses
Statistical inference at the wrong unit of analysis
Scale trend confounded by distillation
Flip rate may partly measure instruction compliance, not explanation quality
Narrow task and metric coverage
ESMP edge cases unspecified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Are LLM self-explanations faithful to what the model actually does?
This paper tests faithfulness counterfactually: the model edits its input so that its own prediction flips, and the edit is faithful if the model's prediction actually changes. Large models (e.g., LLaMA-70B or Qwen-72B) flip their predictions in roughly 0.82-0.96 of cases, while 1B-scale models manage only about 0.03-0.53 depending on task. So faithfulness of counterfactual self-explanations depends heavily on model size and cannot be assumed.
What is Evidence-Supported Modification Precision (ESMP)?
ESMP is a metric introduced by this paper. It aligns the original input with the model's counterfactual edit, then checks what fraction of the edits fall inside spans that humans annotated as decision-relevant evidence (from the ERASER benchmark). It is a precision-style measure: high ESMP means the model concentrated its edits on human-relevant evidence rather than arbitrary tokens.
Does asking an LLM to give a rationale before generating a counterfactual improve the explanation?
It improves different things than you might expect. Rationale-guided prompting produced the most minimal edits and the best alignment with human-annotated evidence (best in 15 of 20 settings each), but it lowered faithfulness - the edits flipped the model's own prediction less often. The paper concludes that plausible-looking, human-aligned explanations are not necessarily faithful to the model's actual decision process.
Which models and datasets were used in this study?
Ten instruction-tuned open-weight models: six Qwen-2.5 models (1.5B to 72B) and four LLaMA-3 models (1B, 3B, 8B, 70B), all run locally with greedy decoding on an NVIDIA H200. Tasks were binary sentiment analysis on ERASER movie reviews (Zaidan and Eisner 2008) and binary natural language inference on e-SNLI, both using ERASER human rationale annotations.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-5 appear only as captions in the extracted text; visual patterns (e.g., per-setting bar charts) could not be independently verified.
- Per-instance sample sizes (N) and any exclusion rates are not stated anywhere in the extracted text, limiting assessment of statistical power.
- The extraction appears complete through Appendix D, but minor truncation of prose between sections cannot be ruled out.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.