Locating and Steering Refusal Beyond Attention
Preethi Carmel Bosco, Gopalakrishnan Srinivasan
cs.LG
Quick verdict
Shows one shared refusal direction across transformer, SSM, recurrent and hybrid models via rigid rotation; rigorous, honest, but gate ties detect-then-refuse.
Summary
The paper asks whether the single 'refusal direction' known from transformer residual streams also exists in state-space models (SSMs), which share no attention mechanism. Using Falcon-Mamba-7B as the SSM anchor, Llama-3.1-8B and Mistral-7B as transformer references, plus RWKV-6 (recurrent) and Zamba2-7B (hybrid), the authors show that a single rigid orthogonal (Procrustes) rotation aligns the two models' representation spaces: a transported harm probe reaches AUROC 1.00 on the target and tracks the target's own probe category-by-category across 16/16 held-out folds, and ablating the transported direction jailbreaks the target (attack success 25.5–79.0% across six transfer pairs) while an orthogonalized random control barely acts. Second, the direction must be estimated at each architecture's 'write site' (attention output, SSM mixer output, RWKV time-mix output), not in the accumulated residual; a matched-magnitude control shows the push location is irrelevant and the estimation site decisive. Third, a detector-triggered gate lowers prefix-injection attack success in all four families (e.g., Falcon-Mamba 98.0%→5.0%, n=200, Wilson CIs) and survives GCG, adaptive, dilution, and probe-evasion attacks. The authors candidly note the gate only ties a trivial detect-then-refuse rule sharing the same detector, the persona-attack fix does not generalize, and RWKV's detector is fully evaded by DAN. Main limitations: code promised only 'upon publication', small samples in some cells (RWKV 40), and defense margin depends on each model's baseline alignment.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A single rigid Procrustes rotation aligns transformer and SSM representation spaces, and the refusal direction transported through it causally controls the target's refusal (ablation raises attack success 25.5–79.0% across six transfer pairs, versus near-zero for orthogonalized random controls).
- The refusal direction must be estimated at each architecture's write site (attention output, SSM mixer output, RWKV time-mix output); the push location is irrelevant while the estimation site decides (matched-magnitude control, Table 7).
- A detector-triggered gate amplifying the write-site direction lowers prefix-injection attack success in all four architecture families (Falcon-Mamba 98.0%→5.0%, n=200) and survives GCG, adaptive, dilution, and probe-evasion attacks.
- The gate adds no defense strength over a trivial detect-then-refuse rule sharing its detector; the architecture-portable contribution is the direction itself.
- Where labels are scarce, a transported harm detector flags 11.4 percentage points more harmful prompts than a probe trained on the same few labels, at a matched 5.0% false-alarm rate.
Strengths
Best-controlled cross-architecture transfer design
The structural claim uses an orthogonal Procrustes rotation that cannot synthesize directions (A.1), a shuffled-correspondence null collapsing transported AUROC to 0.33 (A.7), category-held-out CV where transported and own probes track at Pearson 0.99 across 16/16 folds (Table 3, A.8), a BeaverTails 14-fold replication, and an orthogonalized random control because the ridge map leaks refusal signal up to cosine 0.515 (A.6).
Write-site claim separated from push-location and scale confounds
A matched-absolute-magnitude control (Table 7) shows write-site vs residual pushes within overlapping CIs while the residual-estimated direction degenerates (100.0% ASR on Falcon-Mamba); non-mixing controls (transformer MLP output A.18; SSM skip term D·xt A.19) and a mixer-input proximity control (Table 19: 100% over-refusal at rel=0.5) close the remaining loopholes.
Unusual candor about negative and bounding results
Robustness depth beyond field norm
Weaknesses
Defense contribution is zero by the paper's own showing
Weakest family carries headline weight
Persona attacks remain open and the fix is per-model
Hybrid deployability gap at the causally dominant site
Nothing independently checkable today
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.