Are Stated Reasoning Steps Causally Load-Bearing?
Abhiram Bhupatiraju, Rayan Nyaupane
cs.AI
Quick verdict
Activation patching finds 76.9% of Qwen3-4B's stated CoT steps causally load-bearing vs 88.2% by text editing; rigorous, but narrow synthetic single-model scope.
Summary
The paper asks whether the reasoning steps a model writes are causally responsible for its answers, using targeted activation patching rather than text editing. On a synthetic multi-hop lookup task (2-6 hops) with pseudo-word entities, the authors patch residual-stream activations from a counterfactual run into the token spans where the model states each intermediate step, scoring success as flipping the answer to a by-construction predicted counterfactual entity. For Qwen3-4B, 76.9% +/- 2.8% of stated steps are causally load-bearing at the mid-network layer, against an 11.3% random-position null and an 83% prompt-fact positive control (~96% of achievable effect). On matched items, the standard behavioral text-editing test yields 88.2%, overstating causal faithfulness by 11.4 points (111:14 discordant pairs, McNemar p < 1e-15), with the gap largest on easiest items (20.2 points at 2 hops). Qwen3-1.7B is less faithful (54.8%) and degrades sharply with depth (68.4% at 2 hops to 29.7% at 6). Positive, sanity, and specificity controls support validity. Limitations: fully synthetic task, one model family at two small scales, greedy decoding, early-layer confounds with token identity, and an inherent asymmetry between visible text edits and invisible activation patches. No code or data release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 14 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- For Qwen3-4B, 76.9% +/- 2.8% of stated reasoning steps are causally load-bearing at the most responsive mid-network layer, about 96% of the 83% prompt-fact patch ceiling.
- The behavioral text-editing test on the same items scores 88.2%, overstating causal faithfulness by 11.4 percentage points item-matched (111:14 discordant pairs, McNemar p < 1e-15).
- The behavioral-causal gap is largest on easy items: 20.2 points at 2 hops, narrowing to 6.6 points at 6 hops.
- Causal faithfulness tracks capability: Qwen3-1.7B reaches only 54.8% and collapses from 68.4% at 2 hops to 29.7% at 6 hops, while Qwen3-4B stays flat at 75-79%.
Strengths
Flip-to-target criterion is a genuine methodological advance
Unlike the degradation-based CoT audits the paper cites (refs 4, 6), each patch (Sec 3) carries a by-construction predicted counterfactual answer, so mere perturbation scores as failure. The validator re-deriving gold and counterfactual answers from rendered text (1000/1000 passing) anchors task integrity.
Item-matched calibration of the field's standard metric
Sec 4 applies behavioral text-editing and causal patching to the same items and quantifies the overstatement precisely: 88.2% vs 76.9% at layer 8, 111:14 discordant pairs, McNemar p < 1e-15, with the gap graded by difficulty (20.2 points at 2 hops, 6.6 at 6).
Unusually complete control suite
Interesting capability dissociation
Layer-band localization with honest confound handling
Weaknesses
Asymmetric comparison undercuts the headline framing
Very narrow external validity
Selection-conditioned populations for the capability comparison
Unspecified layer selection and ambiguity in disruption accounting
No code or data release; possible retrieval shortcut
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This preprint (arXiv:2609.27038, cs.AI, 'Under review') asks whether the intermediate steps a model states in its chain-of-thought causally produce the final answer, and how much the field's standard behavioral test mismeasures this. It contributes (Sec 1): (1) the causally load-bearing (CLB) rate, a targeted interchange-intervention metric with by-construction predicted counterfactual targets; (2) an item-matched calibration against behavioral text editing; (3) a two-model capability comparison (Qwen3-4B vs Qwen3-1.7B). Headline results (Sec 4): CLB of 76.9% +/- 2.8% for Qwen3-4B at layer 8 (78.3% at layer 4) versus an 11.3% random-position null and an 83% prompt-fact positive control (~96% of ceiling); behavioral testing yields 88.2% +/- 2.1% on 873 cases, an 11.4-point overstatement (111:14 discordant pairs, McNemar p < 1e-15), graded by difficulty (20.2 points at 2 hops, 6.6 at 6); Qwen3-1.7B reaches 54.8% and collapses from 68.4% (2 hops) to 29.7% (6 hops) while 4B stays at 75-79%.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'causally load-bearing' mean in this paper?
A stated reasoning step is called causally load-bearing if replacing its internal activations with a counterfactual version (from a run where that step resolves to a different entity) flips the model's final answer to the specific counterfactual entity implied by construction. This is stricter than tests that only check whether the answer changes at all.
Do chain-of-thought explanations reflect what the model actually computes?
Partly, according to this paper. On synthetic multi-hop lookup tasks, about 77% of Qwen3-4B's stated steps causally drive the answer, but the common text-editing test reports about 88%, so simple editing experiments appear to overestimate how causally used the written reasoning is - especially on easy items (a gap of about 20 points at 2 hops).
Which models and tasks were tested?
Only Qwen3-4B and Qwen3-1.7B with greedy decoding, on a synthetic in-context knowledge-base task with pseudo-word entities and chains of 2 to 6 lookup hops. The authors note that naturalistic math or code reasoning was not tested, and no other model families were included.
Is code or data available for this paper?
The paper text contains no code or data release statement. Reproduction would require rebuilding the synthetic task generator and a TransformerLens-style activation patching harness for the open-weight Qwen3 models.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 and 2 are available only as caption text; layer-resolved curves and CI widths per hop cannot be inspected directly.
- The metadata abstract is truncated mid-sentence, but the embedded full text covers the corresponding content, so claims were cross-checked against Sec 4.
- Appendix-level implementation details (exact span localization of counterfactual entities, end-alignment code, layer-selection procedure) are absent from the extracted text, limiting methods verification.
- This is an 'Under review' v1 preprint; no peer verification exists yet, and no code/data availability statement is present.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.