Refusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Families
cs.LG · cs.AI · cs.CL
Quick verdict
Claims refusal reads a harm slice, not moral comprehension; methodologically rigorous, but the causal core rests on one weak-refusing model.
Summary
The paper asks what a chat model's refusal decision actually reads, separating it from what the model morally comprehends. Using mean-difference directions across four open-weight models (OLMo-3-7B-Instruct, Qwen2.5-7B, Llama-3.1-8B, GPT-OSS-20B), it shows a rank-3 moral subspace crystallizes during pretraining (checkpoint-to-final cosine 0.999 on OLMo-3) and survives alignment as a single ~40-degree rotation, while the refusal gate is a fresh post-training construction (precursor cosine 0.155) in a low-variance, 8-15-dimensional decision-token bottleneck where refusal and judgment directions are near-orthogonal. The causal core is a nested interchange rank sweep on OLMo-3 with request-twins (n=42 pooled): judgment transfer climbs to 0.66 as the moral basis widens while refusal transfer saturates around 0.24 after rank 3, fitting a one-knob clipped-ceiling model (RMSE 0.023); roughly 76% of refusal's causal input lies outside the rank-16 moral subspace. A cross-model panel finds Llama reads broad moral content (transfer 0.85 vs judgment 0.79) and commits early; GPT-OSS reads harm only correlationally and is behaviorally reversible under prefill; Qwen is indeterminate. Key limitations: the causal result is single-model on the panel's weakest refuser (~17% refusal rate), the refusal-minus-judgment difference CI includes zero at n=42, and the cross-model interpretation is architecture-confounded.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- On OLMo-3, a rank-3 moral subspace crystallizes during pretraining (checkpoint-to-final cosine 0.999) and survives alignment as a single ~40-degree rotation, while the refusal gate is a fresh post-training construction (precursor cosine 0.155) in a low-variance, 8-15-dimensional decision-token bottleneck.
- Causally, on OLMo-3, refusal's interchange transfer saturates at the harm-rank-1 level (0.22-0.24 beyond rank 3) while judgment transfer climbs to 0.66 as the moral basis widens, with about 76% of refusal's causal input lying outside the rank-16 moral subspace.
- Across families, what refusal reads dissociates from how it commits: Llama reads broad moral content (transfer 0.85 vs judgment 0.79 at matched depth) and commits early; GPT-OSS reads harm only correlationally and commits reversibly; Qwen is indeterminate.
- Shallow, easily-removed alignment is explained by a refusal decision that reads a small harm-keyed feature over a narrow channel rather than the model's broad moral representation.
Strengths
Calibrated inference discipline
Every verdict is read against a stated ladder: isotropic floor, covariance-matched rank-matched null (q95), measurement, held-one-out moral-family positive band, persona reference (Table 3); e.g. the 0.155 precursor cosine is read against null q95 0.070, and the band-below-null tell (B.3) correctly invalidates content-projection tests at the 15-dim decision site.
Causal design with replication and negative controls
Nested interchange rank sweep on decision-channel heads with request-twins (23 + 19 pooled, n=42), per-rank subspace purity 0.97-0.99, random-direction null near zero at every rank, and a parsimonious one-knob fit R_refusal(k)=min(0.25, R_judgment(k)) at RMSE 0.023 (Sections 7, C.4-C.5).
Unusually honest epistemic bookkeeping
Exceptional reproducibility infrastructure
Documented instrument self-correction
Weaknesses
Causal core on an atypical, weakly-refusing model
Headline contrast is underpowered at current n
Cross-model table is multiply confounded
Key validity protocol lives in a concurrent self-citation
Conflicting ablation instruments on the same model
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks a mechanistic question: when a chat model refuses, what does the decision read? It separates "what refusal reads" from "what the model morally comprehends," across four open-weight models (OLMo-3-7B-Instruct, Qwen2.5-7B, Llama-3.1-8B-Instruct, GPT-OSS-20B). Four headline claims: (1) a rank-3 moral subspace is pretraining-native, crystallizing to a checkpoint-to-final cosine of 0.999 on OLMo-3 and surviving alignment as one ~40-degree rotation (base-to-SFT 0.999 to 0.757, flat thereafter); (2) the refusal gate is a fresh post-training construction (proto-refusal-to-gate cosine 0.155 vs matched null q95 0.070) in a low-variance channel (percentile 0.0) and an 8-15-effective-dimensional decision-token bottleneck (OLMo 14.7, Qwen 8.6, Llama 10.2, GPT-OSS 12.8); (3) causally, on OLMo-3 only, refusal's interchange transfer saturates at 0.22-0.24 beyond rank 3 while judgment transfer climbs to 0.66, so ~76% of refusal's causal input sits outside the rank-16 moral basis; (4) across families, the read axis dissociates from the commit axis - Llama reads broad (0.85 vs 0.79 at matched depth) and commits early, GPT-OSS reads harm correlationally and commits reversibly (7/7 engage, 6/10 disengage flips), Qwen is indeterminate. The dimensionality-predicts-reversibility reading is explicitly framed as a hypothesis, not a result.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Why is LLM refusal so easy to remove, according to this paper?
The paper argues refusal is a thin post-training control that reads mainly a low-rank 'harm percept' - about 76% of its causal input lies outside the moral subspace on the one causally tested model (OLMo-3) - and is written into a narrow, low-variance decision channel. Because the decision is geometrically separate from the model's broad moral understanding, a rank-one edit can cancel it without damaging comprehension.
Which models does the paper study?
Four open-weight chat models: OLMo-3-7B-Instruct (the causal testbed, with released base checkpoints), Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, and the 20B reasoning mixture-of-experts GPT-OSS, plus reasoning distills in auxiliary harm-direction cells.
Did the paper's main result replicate?
Partially. The OLMo-3 rank sweep was run twice (23 request-twins, then 19 new twins); the pooled n=42 reproduces the harm-saturation shape, but the replication subset alone reads indeterminate, and the direct refusal-minus-judgment difference has a 95% CI of [-0.07, 0.39] including zero at this sample size.
Does the paper claim alignment does not teach models morality?
On what was measured, yes: a moral subspace already crystallizes during pretraining and alignment only rotates it once (~40 degrees) on OLMo-3, while the refusal gate is newly built during post-training. The authors note this causal claim holds on a single model and the cross-family interpretation remains architecture-confounded.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Text extraction was partially truncated (opening of Section 5, body of 8.2, parts of appendices D/F/G); judgments rest on the embedded prose, tables, and appendix fragments.
- Figures are available only as axis labels and captions in text; the visual shape of the Figure 4 plateau could not be inspected directly.
- The pre-registration trail, Zenodo deposit, and GitHub repository could not be verified (no network access); reproducibility ratings follow the paper's own descriptions.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.