</think> Doesn't Stop Reasoning: Analysis of Spurious CoT Termination
Seunghee Koh, Sungjae Choi, Minchan Kwon, Sunghyun Baek, Junmo Kim
cs.CL · cs.AI
Quick verdict
Shows injected </think> often fails to stop LRM reasoning; strong multi-model evidence, but the attention-mechanism causal claim rests on indirect probes.
Summary
Training-free CoT early-exit methods (DEER, DynaSoR) inject the end-of-think token (</think>) mid-reasoning to force the answering phase. The paper finds this injection frequently fails: models continue reasoning-like generation and later regenerate another EoT — 'spurious CoT termination'. Across four LRMs (R1-Distill-1.5B/14B, Qwen3-14B, QwQ-32B) and five benchmarks (GSM8K, MATH-500, AMC 2023, AIME 2024, GPQA-Diamond), EoT regeneration exceeds 20% for QwQ-32B and 15% for R1-Distill-14B, is absent under natural EoT, and rises with compression rate (~40% at 90-100% for R1-Distill-14B+DEER). Regenerated samples show 2-10x longer answering phases, concentrated before the regenerated EoT (correlation with saved reasoning tokens ρ=0.67; Wait markers almost exclusively in that span). The authors hypothesize insufficient attention to the injected EoT and probe it with Exit-token Attention Biasing (EAB): EAB cuts ERR (e.g., R1-Distill-14B+DEER average 16.3%→3.8%) and answer length while mostly preserving accuracy, with effects specific to the exit token and selective to spurious samples. Limitations: evidence is indirect and behavioral, no significance testing is reported, and QwQ-32B sometimes loses accuracy, its regeneration partly reflecting learned generation patterns rather than attention deficits.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Externally injected </think> frequently fails to terminate reasoning: EoT regeneration appears only under injection (0% with natural EoT) and scales with the fraction of reasoning removed by early exit.
- The inflated answering phase is concentrated before the regenerated EoT and behaves like continued reasoning (Wait markers almost exclusively pre-regen, boxed-before-EoT pattern, correlation ρ=0.67 with saved reasoning tokens).
- Increasing attention to the injected EoT (EAB) causally reduces spurious CoT termination and answer length, with effects specific to the exit token (Offset-1 shows the opposite trend) and selective to spurious samples.
- Inserting the explicit think-block delimiter does not by itself guarantee the reasoning-to-answering transition; part of QwQ-32B's regeneration reflects learned generation patterns rather than insufficient attention.
Strengths
Clean factorial characterization of a real failure mode
Three exit cases (natural EoT / early-exit / think-limit) plus compression-rate bins isolate injection as the trigger: natural EoT yields 0% ERR, external injection 2-75% by setting (Table 1, Fig. 2, Table 11), rising to ~40% at 90-100% compression for R1-Distill-14B+DEER.
Converging behavioral evidence that pre-regen spans are displaced reasoning
Table 3/13: pre-regen median Wait count is positive while post-regen is exactly 0 in every setting; Figure 3 correlations (pre-regen vs saved reasoning tokens ρ=0.67, post-regen vs uninterrupted answer ρ=0.71); boxed-before-EoT quantified per model/dataset in Table 15 (END-BOX5 ≈ 0.9-1.0 in pre-regen).
Intervention designed with proper falsification controls
Transparent confound handling
Honest treatment of the disconfirming case
Weaknesses
No statistical inference anywhere
Core correlational result verified on one configuration only
Attention-specific mechanism claim not fully discriminated from alternatives
Non-trivial accuracy regressions under-explained
Coarse behavioral proxy
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Review: '</think> Doesn't Stop Reasoning: Analysis of Spurious CoT Termination'
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is 'spurious CoT termination'?
When an early-exit method injects the </think> token to force a reasoning model to stop thinking, the model may keep generating reasoning-like text (self-corrections, 'Wait') and later emit another </think> of its own. The injected terminator failed to actually end reasoning — the paper names this phenomenon spurious CoT termination.
How often does the injected </think> fail to stop reasoning?
It depends strongly on the model and exit method: averages range from 1-3% for R1-Distill-1.5B and Qwen3-14B, 16-28% for R1-Distill-14B, and 23-34% for QwQ-32B (Table 1), rising to about 40% when early exit removes 90-100% of the reasoning trace (Figure 2).
What is EAB (Exit-token Attention Biasing)?
An inference-time intervention that adds a constant α to the attention score of the injected </think> token for all subsequent generation steps across all layers and heads. Positive α makes the model attend more to the terminator, reducing regeneration and answer length while mostly preserving accuracy; it is presented as a diagnostic probe, not a deployable method.
Does fixing spurious termination improve answer accuracy?
Not consistently. EAB mainly improves efficiency; on some model/method pairs accuracy drops (e.g., DynaSoR+EAB on R1-Distill-14B falls from 74.0 to 70.0 average accuracy, Table 5), which the authors attribute to the quality of the early-exit point rather than the bias itself.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Sections 6.1-6.2 (EAB sweep discussion), parts of Section 5, subsections D.5-D.9 (including possibly Table 14), the tail of the references, and parts of appendix E fall inside marked [...sampling gap...] regions; judgments about those parts rely on surrounding visible text only.
- Claims of absence (e.g., no significance testing) are scoped to the provided text; equivalent material could exist within the sampling gaps, though the gap regions are mostly appendices and references.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.