The Role of Fine-grained Harm Signals in LLM Safety
Soyeon Park (KAIST), Seogyeong Jeong (KAIST), Sunwoo Kim (KAIST), Alice Oh (KAIST)
cs.CL
Quick verdict
Orthogonal category-specific harm directions still carry harmfulness and amplify downstream general-harm alignment; solid empirical study, thin statistics.
Summary
This paper asks whether the category-specific component of harmfulness representations — what remains after removing a shared general harm direction — plays a functional role in LLM safety. Using harmful/harmless prompt pairs from 11 CatQA risk categories, the authors extract mean-difference harm vectors at the final user-instruction token in Gemma-2-9B-IT, Llama-3-8B-Instruct, and Qwen2.5-7B-Instruct, then project out the general harm vector to obtain category residuals orthogonal to general harm at each layer. Three steering experiments follow: harmfulness attribution via a reply-inversion task, refusal induction on benign Alpaca prompts, and change in downstream cosine alignment with the general harm vector. Hate/harassment/violence and adult-content residuals show the largest attribution effects, with category rank order preserved across models (Spearman ρ = 0.89–0.97); refusal induction is more model-dependent (ρ = 0.59–0.80); and most residuals raise downstream alignment above a random-orthogonal-direction control, peaking in middle-to-late layers, with results replicated on a second disjoint prompt sample. Limitations: no significance tests or confidence intervals, differing steering strengths across models (α = 0.25 vs. 0.6) that block absolute cross-model comparison, no random-direction or full-vector ablations in the behavioral tasks, substring-based refusal classification, and no stated code or data release.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Some category residuals (notably hate/harassment/violence and adult content) carry harmfulness beyond category semantics, with a category-wise pattern that is consistent across the three models (§3.1).
- Category residuals can induce refusal of benign prompts, with effects varying more across models than harmfulness attribution does (§3.2).
- Despite being orthogonal to the general harm vector at the injection layer, category residuals usually increase downstream cosine alignment with the general harm representation (§3.3).
- LLM safety analysis should consider fine-grained category residuals alongside the shared general harm representation; orthogonality at one layer does not preclude downstream conceptual amplification (§4–5).
Strengths
Clean conceptual decomposition with a functional test
The paper does not merely document that category harm vectors differ; Eq. (2) defines category residuals by projecting out the general harm vector, then §3.1–3.3 directly test what these residuals do behaviorally and computationally — a step beyond the correlational geometry of Shah et al. (2025).
Cross-model replication of the category ordering
Attribution rank order is highly consistent across three different model families (Spearman ρ = 0.97 Gemma–Llama, 0.92 Gemma–Qwen, 0.89 Llama–Qwen, §3.1), and all findings are re-checked on a disjoint second 100-prompt Alpaca sample (Appendix D, Figure 6).
Appropriate control in the alignment experiment
Honest disclosure of the steering-strength asymmetry
Reproducible formal specification
Weaknesses
No statistical inference anywhere
Behavioral experiments lack control conditions
Steering-strength confound undermines the model-dependence claim
Baseline rates unreported
No code/data release; pair construction underspecified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks a precise question: after removing the shared general-harm direction from each category's harmfulness representation, does what remains — the *category residual* — do anything functionally? Building on Zhao et al. (2025) (harmfulness localized at the final user-instruction token, varying by risk category) and Shah et al. (2025) (substantial shared geometry across categories), the authors construct category harm vectors via mean-difference over matched harmful/harmless prompt pairs (Eq. 1), project out the general harm vector (Eq. 2), and steer three instruction-tuned models (Gemma-2-9B-IT, Llama-3-8B-Instruct, Qwen2.5-7B-Instruct) across 11 CatQA categories. Three claims are advanced: (1) some residuals carry harmfulness beyond category semantics with cross-model-consistent ordering (§3.1); (2) residuals can induce refusal, more model-dependently (§3.2); (3) residuals usually increase downstream alignment with general harm despite injection-layer orthogonality (§3.3). The claims are appropriately scoped to ~7–9B models and one taxonomy, and §6 acknowledges this.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is a 'category residual' in this paper?
It is the part of a risk-category harmfulness direction that remains after subtracting the model's shared general harmfulness direction: the category harm vector with its projection onto the general harm vector removed (Eq. 2), making it mathematically orthogonal to general harmfulness at that layer.
Which models and datasets were used?
Three instruction-tuned LLMs — Gemma-2-9B-IT, Llama-3-8B-Instruct, and Qwen2.5-7B-Instruct — evaluated on the 11 risk categories of the CatQA dataset (100 matched harmful/harmless prompt pairs per category) and 100 benign Alpaca prompts for refusal and alignment experiments.
What is the paper's main finding?
That category-specific harm directions which are orthogonal to general harm still matter: some encode harmfulness and can trigger refusal, and although orthogonal at the layer where they are injected, they usually increase the model's downstream alignment with the general harm representation — so one-layer orthogonality does not mean a direction is conceptually inert.
Is code or data available for this paper?
The extracted paper text contains no code or data availability statement; models (Gemma-2, Llama-3, Qwen2.5) and datasets (CatQA, Alpaca) are public, but the authors' prompt pairs and steering code are not stated to be released.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2–6 appear only as embedded plot descriptions; per-category numeric values beyond those quoted in body text (e.g., exact attribution deltas per category) cannot be verified from the extraction.
- The prompt-pair generation/validation procedure and the random-direction sampling method are absent from the text, so some reproduction details are inferred rather than confirmed.
- Claims about prior work (Zhao et al. 2025, Shah et al. 2025) are taken from the paper's own characterization; their content was not independently verifiable here.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.