Harmfulness Propagation Dynamics: Layer-wise Trajectories of Adversarial Intent in Large Language Models
Noor Islam S. Mohammad, Uluğ Bayazıt
cs.CL
Quick verdict
Paper claims harmful intent rises monotonically across LLM layers; HERALD, a 262KB trajectory moderator, beats guards on jailbreaks — solid, some caveats.
Summary
The paper reports Harmfulness Propagation Dynamics (HPD): across four 7–8B instruction-tuned backbones, the last-token hidden state's projection onto a per-layer LDA harm direction rises monotonically with depth for harmful prompts, while benign prompts stay flat or oscillatory. Building on this, the authors introduce HERALD, an input moderator that compresses the cross-layer projection trajectory into seven geometric features (terminal value, mean, total rise, curvature, monotonicity, onset layer and value) classified by a 288-parameter MLP, needing 262 KB and negligible inference overhead. Trained on WildGuardMix with Ledoit-Wolf-shrunk LDA directions and evaluated zero-shot on eight prompt-harmfulness benchmarks, HERALD averages 89.3 macro-F1 on OLMo2-7B and surpasses four guard models on WildJailbreak (98.4 vs 96.9), though it trails them on ToxicChat and OpenAI Moderation. Ablations attribute the largest gains to the onset-layer feature and LDA direction learning; direction stability across splits exceeds 0.97 cosine similarity, and ablating the peak-monotonicity-layer direction causally reduces refusal rates. Limitations: WildJailbreak shares its WildChat source with the training mix — on a de-confounded 412-prompt subset the lead narrows to 95.7 vs 94.8; the claimed '<1.5% false positives' contradicts Table 14's Alpaca TNRs (93.9–97.1%); the 'beats prior latent methods' comparison uses only generic baselines; code is promised on acceptance, not yet released.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 3
● low 3
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- For harmful prompts, per-layer cosine projections of the last-token hidden state onto LDA harm directions rise monotonically with depth while benign trajectories stay flat, stably across four 7–8B backbones.
- Trajectory shape (onset layer, monotonicity, curvature) carries discriminative information beyond the terminal projection, with the largest gap on adversarial jailbreaks (96.3 terminal-only vs 98.4 full on WJB, OLMo2-7B).
- HERALD surpasses all four tested guard models on WildJailbreak (98.4 vs 96.9 F1) while storing 262 KB and adding roughly 2.6×10^-6 of prefill FLOPs.
- Per-layer LDA directions are stable across random splits (cosine >0.97), and ablating the peak-monotonicity direction causally reduces refusal rates (94.5% to 61.0% on Llama-3.1-8B).
Strengths
Unusually thorough ablation program with the right controls
Tables 3–10 isolate every design choice; the random-direction control (Table 4: 77.2–79.9 F1 vs LDA 86.3–89.3) shows HPD is genuinely directional rather than a projection artifact, and Appendix M adds GRU/1D-CNN sequence baselines on identical projections (GRU 88.9 avg / 97.8 WJB vs 89.3 / 98.4).
Honest negative results and de-confounding effort
The paper reports losing to the best guard on ToxicChat (−4.3) and OpenAI Moderation (−8.4), quantifies suffix-attack degradation (86.3→81.5 at k=200, Table 7), and Appendix S builds a hard-divergent WildJailbreak subset (N=412) narrowing the headline margin from 1.5 to 0.9 F1.
Evidence beyond correlation for the HPD phenomenon
Credible deployability claims with transparent arithmetic
Weaknesses
Internal inconsistency on false positives
Headline jailbreak win rests on an overlapping benchmark
'Prior latent methods' never actually compared
Theory is decorative and comparison asymmetric
Thin error reporting for small margins
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This ICML 2026 Mechanistic Interpretability workshop paper makes one empirical claim and one methodological claim. Empirically, Harmfulness Propagation Dynamics (HPD, Sec. 3): for harmful prompts the cosine projection of the last-token hidden state onto a per-layer LDA harm direction rises monotonically with depth (near-zero below l≈8, pL≳0.4 late), while benign prompts stay flat; validated on 500 sampled prompts per backbone (App. B) across four 7–8B instruction-tuned families. Methodologically, HERALD (Sec. 4): seven trajectory features (pL, mean, total rise, curvature, monotonicity, onset layer/value at a τ90 threshold) fed to a 288-parameter MLP. Headline results: 89.3 average macro-F1 on OLMo2-7B across eight benchmarks; WildJailbreak 98.4 vs 96.9 for the best guard; 2.3–4.1 F1 over latent baselines; terminal-only 87.8 vs 89.3 overall and 96.3 vs 98.4 on WJB (Sec. 6.3, Table 3).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is HERALD and how does it detect harmful prompts?
HERALD is a lightweight input moderator for LLMs. It projects each transformer layer's last-token hidden state onto a per-layer harm direction learned by LDA (with Ledoit-Wolf shrinkage), turning the layer-wise projections into a trajectory. Seven geometric features of that trajectory (terminal value, mean, total rise, curvature, monotonicity, onset layer and value) feed a tiny 288-parameter MLP that outputs harmful/benign. It stores 262 KB of directions for a 32-layer model and adds a negligible ~2.6×10^-6 share of prefill FLOPs.
Does HERALD beat Llama Guard and other safety classifiers?
Partially. On the adversarial WildJailbreak benchmark it reports 98.4 F1 vs 96.9 for the best guard (ShieldGemma-2) and 96.4 for Llama-Guard-3, using an OLMo2-7B backbone. However, it trails the best guard on ToxicChat (−4.3 F1) and OpenAI Moderation (−8.4 F1). Also, WildJailbreak shares its WildChat source with HERALD's training data; on a de-confounded 412-prompt subset the lead narrows to 95.7 vs 94.8.
What is Harmfulness Propagation Dynamics (HPD)?
HPD is the paper's empirical finding that for harmful prompts, the projection of the last-token hidden state onto a per-layer learned harm direction rises monotonically as you go deeper into the network (near-zero in early layers, strongly positive in late layers), while benign prompts show flat or oscillatory trajectories. The claim is that harmful intent is progressively resolved with depth, so the trajectory's shape — not just its final value — is informative.
Is HERALD's code or the trained harm directions available?
Not yet. Appendix Q commits to releasing per-layer LDA direction files (~262 KB each, fp16), a pip-installable herald-moderator package, training code, and evaluation scripts upon acceptance, hosted on HuggingFace Hub under CC-BY 4.0. As of this preprint, nothing has been released.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure panels (Figs. 1–5) are not present in the extracted text — captions only; figure-level judgments here rely on the corresponding tables.
- Guard model names in Appendix P are reproduced as printed; the string 'Aegis-AI-Content-Safety-Defense-2.0' may reflect PDF-extraction noise around the official model name.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.