Inside VLM Chart Reading: Tracing Value Reading from Vertical Bar Charts Across Space and Depth
Tianhao Niu, Qingfu Zhu, Wanxiang Che
cs.CL
Quick verdict
Claims VLMs read bar values via bar-top-dominant, legend-to-prompt handoff pathways; rigorous patching study, but only two 8B models on synthetic charts.
Summary
The paper asks how vision-language models internally combine evidence when reading an exact value from a vertical bar chart. Using controlled counterfactual activation patching on Qwen2.5-VL-7B-Instruct and InternVL3.5-8B, over 500 synthetic charts yielding 5,932 value-reading queries (with success-filtered bundles of 500 queries per condition), the authors run three linked analyses. Single-factor localization shows the changed bar-top region restores far more answer preference (mean restoration 0.783-0.936) than the invariant bar body (0.328-0.490), a gap that persists under token-count-matched restriction; legend- and series-related states also lose local recoverability earlier across depth than bar-geometry and axis-scale states. A handoff analysis finds restoration shifting from legend regions in early layers to prompt-series positions in middle layers, and a held-out reset test shows that resetting prompt-series states blocks part of legend-source rescue (ΔR 0.209 InternVL, 0.426 Qwen) while matched controls stay inactive. A factorial 2×2 geometry-by-scale recombination over 500 families shows both models can combine states from separate donor forward passes, with InternVL insensitive to donor context and Qwen weaker under joint-cross (restoration 0.803 vs 0.867). Limitations include two ~8B models, minimalist synthetic charts, success-conditioned samples that exclude failure cases, and no stated code release.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The changed bar-top region carries most value-specific visual evidence: it recovers a mean 85.7-96.0% of the full-bar restoration effect and far exceeds the unchanged body (0.783-0.936 vs 0.328-0.490), independent of token count.
- Legend- and series-related states become locally unrecoverable earlier across language-model depth than bar-geometry and axis-scale states in both models.
- Legend information is partially mediated by a language-side prompt-series state: a held-out reset test blocks part of legend-source rescue (ΔR 0.209 InternVL, 0.426 Qwen) while matched controls stay inactive.
- Both models can jointly exploit bar-geometry and axis-scale states from separate forward passes; Qwen is more sensitive to donor context (joint-cross R 0.803 vs joint-coherent 0.867; target accuracy 0.908 vs 0.996) while InternVL is not (R 0.977 in both).
Strengths
Well-controlled bar-region contrast
Union/difference/intersection ROIs share identical original-counterfactual image pairs and differ only in the patched support; the difference-over-intersection gap (0.455 [0.430, 0.478] Qwen any-overlap; 0.492 [0.470, 0.513] coarse; 0.445 [0.420, 0.471] InternVL) survives a token-count-matched restriction (0.449/0.480/0.438), cluster bootstrap, and chart-equal reweighting, and holds for 94.6-97.6% of queries.
Bias-resistant mediation design
Source and destination layers are selected on 100 pilot queries with no overlapping source charts against the 400-query held-out set, then frozen before estimation; the prompt-series reset blocks part of legend-source rescue (ΔR 0.209 InternVL, 0.426 Qwen) while matched controls are inactive (prompt-label ΔR 0.001/0.035; target-bar and control-bar resets ≤0.001).
Statistical hygiene above field norm
Transparent data auditing
Careful epistemic framing
Weaknesses
Narrow counterfactual parametrization
Two models, one scale class
Success-conditioned scope
No code or data release stated
Presentation polish
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a mechanistic-interpretability study of how two open-weight vision-language models — Qwen2.5-VL-7B-Instruct (28 language layers, 48×48 merged image-token grid) and InternVL3.5-8B (36 layers, 16×16 grid) — read the exact integer value of a queried bar in synthetic vertical bar charts. Using controlled counterfactual activation patching (500 charts → 5,932 queries; 500 queries per condition across nine conditions), the paper advances four claims: (1) the changed bar-top region carries most value-specific evidence, far out-restoring the unchanged bar body; (2) legend- and series-related states lose local recoverability earlier across depth than bar-geometry and axis-scale states; (3) legend information is partially mediated by language-side prompt-series states; (4) geometry and scale states from separate forward passes can be recombined, with Qwen more donor-context-sensitive than InternVL. The stated scope is deliberately narrow: minimalist synthetic charts and success-conditioned samples.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is activation patching and how is it used in this paper?
Activation patching swaps selected internal states — here, projected image tokens or specific prompt-token activations at a given layer — between an original chart and a counterfactual copy differing in one known factor, then measures how far the model's answer preference moves back toward the original answer. The paper reports this as a normalized restoration score R, where 1 means full recovery of the original preference and 0 means no change.
Which models were analyzed and on what data?
Qwen2.5-VL-7B-Instruct (28 language layers, 48×48 merged image-token grid) and InternVL3.5-8B (36 layers, 16×16 grid), on 500 synthetically generated vertical bar charts with meaningless pseudoword labels, producing 5,932 value-reading queries; 500 queries per counterfactual condition, each answered correctly by both models on both endpoints.
Do the findings apply to real-world charts like ChartQA?
Not directly. The charts are synthetic, fixed-layout, integer-valued, and free of grids, decorations, and OCR noise; the axis scale is only ever doubled and bar heights change by exactly one tick. The authors state that conclusions apply to the clean, correctly-answered regime and leave natural-chart transfer to future work.
What is the main takeaway of the paper?
In two open-weight VLMs, value-reading evidence concentrates in the changed bar-top pixels rather than the bar body; legend identity appears to be recoded into language-side prompt-series positions in middle layers; and downstream computation can recombine bar-geometry and axis-scale information even when it comes from different forward passes.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures appear in the extraction only as fragmented axis-tick and legend labels; curve shapes and error-band details could not be visually inspected, so figure-dependent judgments rest on in-text numbers.
- Inline equations were flattened by PDF extraction (e.g., superscripts in Eq. 1); metric logic was verified from surrounding prose and the Appendix B definitions rather than from the rendered math.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.