Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models
Shardul P. More, Tanuja S. Pawar
cs.CL · cs.LG
Quick verdict
Proposes attention dispersion as zero-shot hallucination signal; gains modest but significant; the headline metric's own contribution is never isolated.
Summary
The paper proposes tracking the standard deviation of layer-wise attention entropy at each generation step (step_attn_std) as a zero-shot, logit-free indicator of reasoning failure in LLMs. Motivated by the claim that RLHF miscalibrates output confidence, the authors compute Shannon entropy of attention distributions per layer (averaged over heads), then measure its dispersion across depth, aggregating over the sequence (mean_attn_std, mean/max_attn_entropy, attn_tau). Logistic-regression classifiers are trained on these features versus output-entropy baselines (mean/max_out_entropy, out_tau) to predict whether the final answer on GSM8K or MATH-500 is incorrect, using repeated stratified 5-fold CV (10 repeats) and ROC-AUC. Attention features beat output baselines in all three settings — Qwen2.5-1.5B on GSM8K (+0.0093), 1.5B on MATH-500 (+0.0486), 3B on MATH-500 (+0.0759) — with 95% CIs excluding zero and one-sided Wilcoxon p-values from 1.66e-3 to 5.65e-8; combined features perform best. Feature weights show mean and max attention entropy acting in opposite directions. Main limitations, largely self-disclosed: 'hallucination' is operationalized as incorrect final answers, only Qwen2.5 (1.5B/3B) and math tasks are tested, the headline setting has n=254, parsing-coverage selection effects exist, no code release, and no empirical comparison against sampling-based or probe-based baselines despite motivating the work against them.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Temporal dispersion of layer-wise attention entropy (step_attn_std) is a zero-shot, logit-independent indicator of reasoning failure in LLMs.
- Attention-based features outperform output-entropy/confidence baselines in all three tested settings, with AUC gains of +0.0093 to +0.0759.
- Attention dispersion carries information orthogonal to output confidence, as evidenced by the Combined model achieving the highest AUC.
- Gains grow with model scale and task difficulty, though the authors concede these two factors were not independently controlled.
Strengths
Simple, fully specified, logit-free metric
step_attn_std (Section 3.2) is a closed-form, unsupervised statistic of attention entropy across layers, computed in a single forward pass — far cheaper than sampling-based uncertainty (Semantic Entropy needs 5-10 generations) and independent of output calibration.
Consistent, statistically reported gains
Table 1 shows positive deltas in all three settings (+0.0093 to +0.0759 AUC) with 95% CIs excluding zero and Wilcoxon p-values from 1.66e-3 to 5.65e-8, under repeated stratified 5-fold CV with 10 repeats.
Complementarity evidence
Unusually candid limitations
AI-assistance transparency
Weaknesses
Headline metric's contribution never isolated
'Hallucination' mislabeled
No comparison against the methods it positions against
Weak absolute performance and selection effects
Scale/complexity confound in the trend narrative
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is 'attention dispersion' in this paper?
For each generated token, the authors compute the Shannon entropy of the attention distribution in every transformer layer (averaged over heads), then take the standard deviation of those entropies across layers. High dispersion means attention focus varies erratically with depth, which the paper proposes as a signal of unstable or failing reasoning.
Does this method detect hallucinations without retraining the model?
It requires no probe training on model internals and no multiple sampling runs: features come from a single forward pass and feed a small logistic-regression classifier. However, it needs white-box access to internal attention weights, so it cannot be applied to closed APIs, and a classifier is still fitted on labeled data for the evaluation.
How large is the improvement over baselines?
ROC-AUC improved by +0.0093 (GSM8K, 1.5B model), +0.0486 (MATH-500, 1.5B), and +0.0759 (MATH-500, 3B) over output-entropy baselines, all statistically significant. Absolute AUCs remain modest (roughly 0.58-0.70), and the baselines are output-logit metrics, not stronger sampling-based methods like Semantic Entropy.
Which models and datasets were used?
Qwen2.5 1.5B and 3B parameter models on GSM8K (grade-school math word problems) and MATH-500 (competition math subset). In this paper 'hallucination' means the final boxed math answer was incorrect — it does not cover factual fabrication in open-domain question answering.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 and 2 are available only as captions; ROC curve shapes and coefficient magnitudes/signs beyond the prose description could not be analyzed.
- Per-setting sample sizes other than n=254 (3B/MATH-500) are not reported in the extracted text, limiting assessment of statistical power for GSM8K and the 1.5B setting.
- Prompt templates, decoding parameters, and layer/head counts are absent from the text, so exact numerical reproduction is not possible from the paper alone.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.