A Unified Mechanistic Analysis of Knowledge- and Safety-Based Refusals
Yuri Son, Seunghee Kim, Hyuhng Joon Kim, Taeuk Kim
cs.CL
Quick verdict
Finds KR/SR refusals share one direction with asymmetric transfer; strong controlled evidence, though behavior-filtered data may bias shared-mechanism claims.
Summary
The paper asks whether knowledge-based refusal (KR) and safety-based refusal (SR) in LLMs share a mechanism. The authors build 213 matched contrastive quadruples (KR, SR, KC, SC) sharing topic and template, generated with GPT-4o-mini and filtered by rules, GPT-5 judgment, and behavioral verification across six models; they also sequentially fine-tune Llama-3-8B, Qwen2.5-7B, and Gemma-2-9B with knowledge-then-safety (Seq-KS) objectives to control training history. Analyses of final-prompt-token hidden states show: (1) KR−KC and SR−SC mean shift directions align at 0.699–0.794 cosine at the peak layer, with a common component projecting 0.92–0.95 and persistent 0.32–0.39 type-specific residuals; (2) probe transfer is asymmetric — SR→KR 0.80–0.86 versus KR→SR 0.58–0.63 (gap 0.22–0.27) in every model; (3) type-specific signals diverge only in upper layers, with logit lens separating epistemic from normative vocabulary. Steering with the estimated directions raises refusal rates and shifts refusal rationales. Robustness checks add Qwen3.5-9B, Gemma-3-12B, the reversed Seq-SK order, and an XSTest-derived 125-example set. Limitations: the behavioral filter removes prompts where models diverge, potentially inflating shared-mechanism conclusions; steering evaluation uses a lexical refusal classifier with weak effects on Gemma*; the commit-then-specify account remains correlational. No code or dataset release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- KR and SR share a dominant common refusal direction: peak-layer shift alignment of 0.699–0.794 with the common component projecting at 0.92–0.95.
- Cross-type probe transfer is persistently asymmetric: SR→KR reaches 0.80–0.86 while KR→SR stays at 0.58–0.63 across all six models and all robustness settings.
- Type-specific specialization emerges sharply in upper layers, with KR aligned to uncertainty/knowledge vocabulary and SR to safety/policy vocabulary (logit lens, Figure 8).
- Sequential knowledge-then-safety tuning (Seq-KS) best preserves general utility while instilling both refusal types (e.g., Llama Seq-KS MMLU 61.84 vs Seq-SK 46.64, Table 4).
- Refusal behaves as a commit-then-specify process: a shared early commitment to refuse, then late-layer type-specific grounding in epistemic or normative reasons.
Strengths
Matched contrastive design with the template confound directly tested
The 213 quadruples (§3.1) hold template, topic, and anchors fixed; Figure 3 shows Δsim = sim(KR,SR) − sim(KC,SC) is negative at every layer in all six models, ruling out shared input structure as the source of refusal-pair similarity.
Training-history confound controlled and the choice empirically justified
Sequential tuning (§3.2, Appendix B, Table 4) compares Seq-KS, Seq-SK, and Merged; Seq-KS preserves utility (Llama MMLU 61.84/GSM8K 66.57 vs Seq-SK 46.64/35.31), and the RQ2 asymmetry persists under both orders (Table 8) and in base models (Figure 12).
Converging multi-method evidence for each research question
Broad robustness program
Above-average statistical candor
Weaknesses
Behavioral-verification filter creates selection circularity
Inferential statistics described but not reported
Cue-salience alternative to the asymmetry is not excluded
Behavioral validation of directions is uneven and under-quantified
No code or data release stated; dataset depends on proprietary APIs
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper tackles a cleanly posed question at the intersection of interpretability and alignment: do knowledge-based refusals (declining unanswerable requests) and safety-based refusals (declining unsafe requests) arise from a shared internal mechanism? It claims three findings — a shared refusal direction (RQ1), asymmetric cross-type transfer with SR→KR stronger (RQ2), and late-layer type-specific specialization (RQ3) — packaged into a commit-then-specify account. Scope is single-turn, open-weight models: Llama-3-8B-Instruct, Qwen2.5-7B-Instruct, Gemma-2-9B-it plus tuned variants, with robustness on Qwen3.5-9B and Gemma-3-12B. Claims are mostly stated at the strength the evidence supports, and the Limitations section is candid about open-weight-only, single-turn, controlled-set scope.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do LLMs refuse unanswerable questions and unsafe questions using the same internal mechanism?
This paper finds substantial overlap: knowledge-based refusal (declining what the model cannot know) and safety-based refusal (declining harmful requests) share a common refusal direction in the model's hidden states, with peak-layer cosine alignment of 0.699–0.794 across six models. But the overlap is asymmetric: a boundary learned from safety-refusal prompts predicts knowledge-refusal prompts at 0.80–0.86 accuracy, while the reverse transfers at only 0.58–0.63. Each type also develops distinctive features in upper layers — epistemic vocabulary for knowledge refusal, policy vocabulary for safety refusal.
What is the commit-then-specify account of refusal?
The paper proposes that refusal unfolds in two stages: early-to-middle layers contain a shared mechanism that first commits the model to refusing, and later layers specify the grounds — epistemic (the question is unanswerable) for knowledge-based refusal or normative (the request is unsafe) for safety-based refusal. The evidence is representational (direction alignment, probes, logit lens) plus activation steering; the authors themselves note it is a geometric diagnostic, not a proven causal circuit.
What dataset did the paper introduce and is it public?
213 matched contrastive quadruples (KR, SR, KC, SC): each quadruple shares one topic and question template, pairing a knowledge-refusal query (fictional or unverifiable target), a safety-refusal query (real sensitive target), and two minimal-edit controls that should be answered normally. Prompts were generated with GPT-4o-mini, judged by GPT-5, and behaviorally verified on six open-weight models; a 75-item subsample was human-checked (Cohen's κ = 0.82). The provided text contains no statement releasing the dataset or code publicly.
Which models were analyzed and could I reproduce the results?
Llama-3-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-it, each also in a sequentially refusal-tuned variant, with robustness checks on Qwen3.5-9B and Gemma-3-12B. The core analyses (difference-in-means directions, logistic probes, logit lens, activation steering) need only forward-pass access to open-weight models, so they are reproducible on a single GPU. The obstacle is the exact 213-quadruple prompt set, which is not stated to be public and was built with GPT-4o-mini/GPT-5.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Text embedded within figures is garbled by PDF extraction (axis labels and tick values interleaved in Figures 3–15); all numeric findings cited in this review come from in-text sentences and Tables 1–10, which extracted cleanly.
- The provided text runs complete through the references and Appendices A–E (ending at Figure 25), so the absence of a code/data availability statement reflects the full document, not truncation.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.