Evolve.Pub
Sign inScan your paper
arXiv paper· machine learning interpretability (vision-language models)· empirical study

Visual Attention Faithfulness in Vision-Language Models is Heterogeneous

Xurui Song, Weishi Wang, Zhongqi Yue, Kuluhan Binici, Tao Bai, Hongxin Shao, Daniel Dahlmeier, Jun Luo

View original paper ↗arXiv:2609.00830

cs.CV · cs.AI

Quick verdict

Perturbation study finds VLM visual attention faithfulness heterogeneous (3 modes); novel but small n and manual mode partition temper the conclusions.

Summary

The paper asks whether attention weights in vision-language models (VLMs) faithfully reflect which visual tokens drive predictions, extending the NLP attention-faithfulness debate to the visual modality. The authors aggregate attention across all layers and heads onto visual tokens in Qwen2.5-VL-7B and InternVL2.5-8B, zero-ablate ranked tokens at the first decoder layer, and measure normalized comprehensiveness (necessity) and sufficiency gap on VQAv2 (90 samples, 85 valid), VRDU (60) and ChartQA (100). On VQA, attention-ranked top-k tokens achieve the best scores (Comp 0.813, Sgap 0.469) versus human-annotated GT regions (0.505/0.846) and random (0.502/0.599). A two-stage Otsu partition plus manual refinement yields three modes: Faithful-Sufficient (32.9%), Faithful-Distributed (47.1%), Non-Focal (20.0%). Human GT regions satisfy comprehensiveness in only ~60% of cases, indicating models rely on regions beyond human-annotated evidence. On document IE, Qwen shows Mode B (Sgap about 0.947) while InternVL shows Mode A (Sgap 0.189 at 10% keep), so mode assignment is architecture-dependent; per-sample mode overlap between the two models is only 50.6%. Limitations: small evaluation sets, manual refinement of the mode partition, a single attention-aggregation scheme, analysis restricted to correct answers on IE, and no stated code release.

Review complete

Assessment at a glance

machine learning interpretability (vision-language models)

4

risk flags identified

● high 0

● medium 3

● low 1

Reproducibility readinessmedium
Code & data availablenot stated
Review confidencehigh
AI text signal0/20 passages flagged
Deep research viableCode-reproducibleNeeds GPU

5 strengths · 5 weaknesses · 5 author questions

AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.

Levels reflect verification complexity — not a quality score. Evolve never rates your paper.

Key claims under review

  • Visual attention faithfulness in VLMs is heterogeneous: valid VQA samples fall into three processing modes - Faithful-Sufficient (32.9%), Faithful-Distributed (47.1%) and Non-Focal (20.0%).
  • Attention-ranked top-k visual tokens are the most causally important selection tested, reaching comprehensiveness 0.813 versus 0.505 for human-annotated GT regions and 0.502 for random ablation on Qwen2.5-VL-7B VQA.
  • Human-annotated ground-truth regions satisfy comprehensiveness in only about 60% of cases, so human annotations are an incomplete proxy for what VLMs causally rely on.
  • Mode membership is architecture-dependent: on document IE Qwen2.5-VL behaves as Faithful-Distributed (Sgap about 0.947) while InternVL2.5 behaves as Faithful-Sufficient (Sgap 0.189 at 10% keep), with only 50.6% per-sample mode overlap on VQA.

Strengths

  • Genuine gap, first systematic causal test

    Extends the settled NLP attention-faithfulness debate (Jain and Wallace 2019 vs Wiegreffe and Pinter 2019) to the visual modality; Section 2 correctly identifies that token-pruning work (Chen 2024a; Shang 2025) assumes rather than tests attention sparsity.

  • Controlled perturbation design

    Four strategies (Overall Top-k, GT Object, Non-GT Top-k, Random) ablate the same token count k set to the GT region size (Section 4.1), so contrasts like Table 1 isolate selection quality rather than ablation volume.

  • Transparent, defensible intervention choices

  • Honest cross-model reporting

  • Actionable GT-divergence analysis

Weaknesses

  • Mode partition is partly manual and unblinded

  • Small evaluation sets without inferential statistics

  • No sensitivity analysis for the attention signal

  • Analysis restricted to correct predictions

  • No code or annotation release stated

Full Review Report

$1.99 · one-time

Unlock the complete preliminary analysis Evolve produced for this paper.

Report preview

Scope & Claims

The paper asks whether attention over visual tokens faithfully indicates what VLMs causally rely on, transplanting the NLP attention-faithfulness debate (Jain and Wallace 2019 vs Wiegreffe and Pinter 2019) to the visual modality. Four claims: (1) the first systematic causal evaluation of visual attention faithfulness in VLMs; (2) three heterogeneous processing modes; (3) human GT regions satisfy comprehensiveness in only about 60% of cases; (4) mode membership is task- and architecture-dependent. Scope: Qwen2.5-VL-7B/3B and InternVL2.5-8B/4B on VQAv2 (90 samples, 85 valid after excluding 5 whole-image GT cases), VRDU (60 correctly-answered samples) and ChartQA (100). Within this scope the claims are directionally supported, but the evidential weight is modest relative to the generality of the title.

The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.

  • All 5 strengths & 5 weaknesses, analyzed in detail
  • 4 risk flags with severity, section & reviewer questions
  • Methods & statistical evidence assessment
  • 5 questions a peer reviewer would ask
  • Full six-dimension evaluation report & recommendations

One-time purchase · this paper only

One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.

Frequently asked questions

What does attention faithfulness mean in this paper?

It means the image regions (visual tokens) receiving the highest attention are also the ones that causally matter for the model's answer. The authors test this by deleting or keeping those tokens and measuring how much the prediction probability changes.

Which models and datasets were studied?

Open-weight vision-language models Qwen2.5-VL (3B and 7B) and InternVL2.5 (4B and 8B), evaluated on VQAv2 visual questions (90 samples, 85 valid), VRDU document information extraction (60 samples) and ChartQA chart questions (100 samples).

What are the three attention processing modes?

Faithful-Sufficient: the top attention tokens are both necessary and sufficient for the answer (32.9% of valid VQA samples). Faithful-Distributed: they are necessary but broader image context is still needed (47.1%). Non-Focal: no localized attention region is necessary even though the image still matters (20.0%).

Does this paper prove attention is not explanation for VLMs?

No, it shows a mixed picture: attention-ranked tokens were the most causally important selection tested (comprehensiveness 0.813 on VQA, beating human-annotated regions at 0.505), but in about 20% of samples no attention region was necessary, and human-annotated regions satisfied comprehensiveness in only about 60% of cases.

About this review

This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.

  • Figure-internal content (scatter densities in Figures 2, 4, 7; heatmap geometry in Figures 3, 6, 8) cannot be verified from text extraction; judgments about those figures rely on captions and in-text numbers only.

Go deeper on this paper

Wrote a paper? Stress-test it before reviewers do.

Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.

Scan your paper
Visual Attention Faithfulness in Vision-Language Models is Heterogeneous | Evolve.Pub