Does Deeper Reasoning Compromise Alignment? Revealing and Mitigating of Alignment Collapse in Large Reasoning Models
Yu-Hang Wu, Yu-Jie Xiong, Henghua Zhang, Bairui Zhang, Jia-Chen Zhang, Shaohua Li
cs.AI · cs.CL
Quick verdict
Argues deep reasoning erodes alignment via attention dilution; RT attack and RRA defense are suggestive, but the paper conflates robustness with alignment.
Summary
The paper claims that extending chain-of-thought reasoning destabilizes alignment in Large Reasoning Models, a phenomenon it names Alignment Collapse. Using Qwen3-8B/14B, whose reasoning budget switches between L=0/2048/4096, the authors evaluate on AIME2024 and LogicAsker under two distractor perturbations (an inserted irrelevant sentence; scenario nesting) and define the Alignment Loss Rate (ALR) as the relative accuracy drop. ALR rises monotonically with depth — Qwen3-14B goes from 9.2% to 33.1% on AIME2024 — although effects on GLM4.6 (0.1%→0.3%) and Claude-Haiku-4.5 (3.1%→5.4%) are marginal. Exploiting this, Reasoning Trap (RT) concatenates a reasoning-trigger template with existing jailbreaks (FlipAttack, PAP, ArtPrompt) on AdvBench: Qwen3-8B's rejection rate under FlipAttack falls from 92.31% to 57.42% when reasoning is enabled, and to 29.03% with FlipAttack+RT. The mechanism is attributed to Attention Dilution: softmax competition plus RoPE distance decay shrinks attention to the original instructions as reasoning grows. The defense, Reasoning Residual Alignment (RRA), re-appends the input ([X;Y;X]) and recovers several points (e.g., 57.42→64.57), though a fixed safety reminder outperforms it on AdvBench. Key limitations: 'alignment' is operationalized as task accuracy under distractors; the safety judge and RT template are unspecified; trial counts are inconsistent (four vs eight); and no length-matched control isolates reasoning itself.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Increasing reasoning budget L monotonically raises the Alignment Loss Rate under perturbations across Qwen3-8B/14B, with smaller but same-signed effects on Claude-Haiku-4.5 and GLM4.6.
- Concatenating a reasoning-trigger template (RT) with existing jailbreaks sharply reduces rejection rates, e.g., Qwen3-8B's FlipAttack RSR falls from 92.31% (standard) to 57.42% (reasoning) to 29.03% (FlipAttack+RT).
- Attention Dilution — softmax competition over a growing reasoning prefix plus RoPE distance decay on the original instructions — is the mechanism behind the collapse.
- RRA, re-injecting the input as [X; Y; X], is a training-free defense that restores part of the lost robustness and generalizes better than a fixed safety reminder on reasoning benchmarks (Tables 11-12).
Strengths
Genuinely controlled depth manipulation
Qwen3-8B/14B switch reasoning budget L∈{0,2048,4096} on identical weights (§3.1), isolating the depth variable that most LRM studies confound with model choice; clean accuracy rises (8B: 20.83%→31.67% on AIME2024, Table 1) while fragility also rises, a clean double dissociation.
Multi-model, multi-dataset triangulation
The ALR trend is replicated on AIME2024 and LogicAsker (Tables 1, 3) and extended to API models Claude-Haiku-4.5 (Table 2) and GLM4.6, with safety results spanning Qwen3, Llama2, DeepSeek-V3/R1 (Tables 4-5) — beyond the single-family evidence common in this genre.
Unusually honest defense ablation
Mechanism tied to architecture, with arithmetic that checks out
Attack actually does what it claims behaviorally
Weaknesses
Construct conflation: distractor robustness is not alignment
Two load-bearing components unpublished
Inconsistent statistics and unstated sample sizes
Missing control isolates nothing about 'reasoning' per se
Overclaimed novelty and generalization, plus editorial errors
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether extended chain-of-thought reasoning weakens alignment in Large Reasoning Models (LRMs) and makes four claims: (1) a new phenomenon, Alignment Collapse — deeper reasoning increases relative accuracy loss under perturbations, quantified by a proposed Alignment Loss Rate (ALR); (2) a jailbreak, Reasoning Trap (RT), concatenating existing attacks with a reasoning-inducing template; (3) a mechanism, Attention Dilution, formalized via softmax competition and RoPE distance decay (Appendix C); (4) a training-free defense, Reasoning Residual Alignment (RRA), re-injecting the input as [X; Y; X]. Section 1's claim to be 'first' sits uneasily beside the paper's own citation of Lou et al. (2025), which already unveiled 'safety alignment collapse' in multimodal LRMs.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Alignment Collapse in large reasoning models?
As defined in this paper, it is the finding that when a reasoning model gets a longer chain-of-thought budget (L=2048 or 4096 vs none), its accuracy on the original task degrades more severely under distractor perturbations. The paper quantifies this with the Alignment Loss Rate (ALR): Qwen3-14B's ALR on AIME2024 rises from 9.2% without reasoning to 33.1% at L=4096, though effects on GLM4.6 and Claude-Haiku-4.5 are much smaller.
What is the Reasoning Trap (RT) jailbreak and how well does it work?
RT is a prompt-level attack that appends a template pushing the model to produce long explicit reasoning before answering, combined with an existing jailbreak (FlipAttack, PAP, or ArtPrompt). On AdvBench, it cut Qwen3-8B's rejection rate for FlipAttack from 92.31% (no reasoning) to 29.03% (FlipAttack+RT). It did not move rejection rates at all for Llama2-7B/13B under FlipAttack, so it is not universally effective. The exact template is not published in the paper.
What is Attention Dilution and the RRA defense?
The paper argues that as reasoning tokens accumulate, softmax attention is spread over more positions and RoPE's distance-based decay weakens scores on the original instructions, so the model effectively 'forgets' its constraints. RRA (Reasoning Residual Alignment) counters this by re-appending the original input after the reasoning trace — context [X; Y; X] — restoring several points of refusal rate without any training. Notably, the paper's own ablation shows a fixed safety reminder outperforms RRA on AdvBench (63.60 vs 61.78 average RSR for Qwen3-8B).
Is code or data available for this paper?
No code or data release is mentioned. The datasets (AIME2024, LogicAsker, AdvBench) and most models (Qwen3, Llama2, DeepSeek) are public, and the perturbation-generation prompts appear in Table 7, but the safety judge setup and the RT reasoning-trigger template are not specified, which blocks exact replication of the safety results.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure content (Figs 3, 4, 9) survives only as fragmented axis labels in the extraction; visual trend claims were cross-checked against table values where possible, not against the plots themselves.
- The reasoning-trigger template T is absent from the entire provided text including all appendices — a content gap in the paper as extracted, not an extraction truncation.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.