Do VLMs Share Safety Neurons Across Modalities?
Jiaxuan Li, Jiahao Zhang, Duc Minh Vo, Huy H. Nguyen, Pride Kavumba, Koki Wataoka
cs.LG
Quick verdict
Across 10 VLMs, safety localizes to ~88 sparse text neurons while visual safety appears diffuse; methodologically careful, but 'diffuse' partly reflects the detection design.
Summary
This paper asks whether vision-language models (VLMs) use the same localized safety neurons for image-borne harm as for text. The authors build a two-stage detector—behavioral probing on the model's own refuse/comply labels, then mean-activation patching certified against a 300-neuron control null under BH-FDR—wrapped in three rounds of iterative ablation to expose self-repair backups. They also introduce ViSafe-Detect (~4.8K images under one neutral query) and ViSafe-Eval (472 high-consensus harmful intents x 8 image conditions x 3 query types, ~11.3K samples per model). Across 10 VLMs (2B-32B, six families), text safety localizes to 48-130 neurons (<0.01%) whose ablation cuts IID refusal by 9-33 points and OOD refusal by 2-21 points with capability intact; visual stimuli yield only 0-9 neurons with sign-inconsistent, non-transferable effects, and GEVD subspace ablation saturates by k=5 for text but keeps growing to k=50 for visual. The paper is candid about limits: detection data is 82.5% BeaverTails-V, evaluation uses a single GPT-4o judge, intents were pre-filtered for high text refusal, and only the post-fusion LLM backbone is tested—so 'visual safety is diffuse' is partly a statement about what this neuron-localizing pipeline can find.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Text safety in VLMs localizes to roughly 88 neurons per model on average (range 48–130, <0.01% of all neurons), whose ablation reduces IID refusal by 9–33 percentage points and OOD refusal by 2–21 points.
- Ablating these text safety neurons is the only intervention that consistently and substantially reduces refusal across all image conditions, including harmful-image-only settings where visual-neuron ablation has no systematic effect.
- Visual safety is diffuse at the single-neuron level: the same pipeline recovers only 0–9 visual neurons per model with sign-inconsistent, non-transferable effects.
- Representation-level GEVD analysis mirrors the gap: text safety saturates within ~5 ablated directions while visual safety keeps growing through 50 directions across the models with measurable baselines.
- This asymmetry is a post-fusion property of the LLM backbone, appearing in both adapted and natively multimodal architectures, implying visual-side alignment needs signals applied to visual representations directly.
Strengths
Causally disciplined detection pipeline
Sec. 3.4 replaces zero-ablation with mean-activation patching (patched/clean L2 ratio >0.995 across all 10 models, App. A.1), standardizes effects against 300 layer-stratified control neurons, and applies BH-FDR at alpha=0.05 with sign-agreement requirements — a statistical standard rare in neuron-localization work.
Iterative ablation addresses self-repair
Sec. 3.5/5.2: single-pass detection recovers only 38–78% of neurons; three rounds raise per-model counts (e.g., Qwen2.5-VL-7B: 76→22→7), directly targeting the hydra-effect undercounting that plagues one-shot methods.
Modality-isolated factorial benchmark design
Cross-architecture robustness and honest self-critique
Weaknesses
Detection-set concentration undermines the 'diffuse visual safety' null
Curated high-consensus intents limit generality
Single-judge evaluation without variance estimates
No matched random-neuron ablation control for the headline causal asymmetry
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether VLM backbones reuse the localized safety neurons known from text-only LLMs when refusing image-borne harm. It claims three results across 10 VLMs (2B–32B, six families, Table 2/4.1): (i) text safety localizes to 48–130 neurons (~0.01%, Table 3; mean ≈88) with IID refusal drops of 9–33 points and OOD drops of 2–21; (ii) text neurons are the dominant refusal pathway — ablating them cuts judge-rated refusal by ~43% on average under direct harm and ~20% under procedural-query-plus-harmful-image, while visual-neuron ablation moves refusal by ~1% and a sign-inconsistent ~+5% respectively (Table 5); (iii) visual safety is diffuse at the single-neuron level (0–9 neurons, Table 4) and needs ≥50 GEVD directions where text saturates at ~5 (Tables 6, G.1). The claims are carefully scoped in Limitations to the refusal-active regime and the post-fusion backbone.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do vision-language models use the same safety neurons for images as for text?
According to this paper, no. Across 10 VLMs, refusing harmful text localizes to a small set of about 48–130 neurons per model (under 0.01% of all neurons), while the same detection pipeline applied to harmful images finds only 0–9 such neurons per model, with inconsistent and non-transferable effects. The authors conclude visual safety is diffuse at the single-neuron level and instead spread over many representation directions (at least 50 in their subspace analysis), whereas text safety fits within about 5 directions.
What are ViSafe-Detect and ViSafe-Eval?
They are the two modality-isolated benchmarks introduced in the paper. ViSafe-Detect has roughly 4,800 images (2,417 harmful, 2,417 safe), each paired with one fixed neutral query ('The image shows a scene of an activity. List steps in detail to perform this activity.'), and is used to discover safety neurons. ViSafe-Eval crosses 472 harmful intents with 8 image conditions (no image, blank, noise, safe image, harmful image, and three rendered-text styles) and 3 query types, about 11,300 samples per model, to attribute refusal to the image, the query, or their interaction.
Can you disable a VLM's safety refusal by ablating a few neurons?
The paper shows that in research settings, ablating the roughly 88 identified text safety neurons (fewer than 0.01% of all neurons) reduces refusal rates by 9–33 percentage points in-distribution and 2–21 points out-of-distribution, while general capabilities (perplexity, MMLU, MMMU) stay largely intact. The authors deliberately do not release per-model neuron coordinates or turnkey ablation code, citing misuse risk.
What are the main limitations of this study?
The authors list four: only the post-fusion LLM backbone is tested (vision encoder and projection layers are unexamined); 82.5% of the detection harmful images come from BeaverTails-V; the evaluation uses 472 harmful intents pre-filtered for high text-refusal consensus; and refusal is scored by a single judge model (GPT-4o), which may bias absolute rates though within-model comparisons remain valid.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Fig. 4 case-study content and Fig. 2/3 diagram details are conveyed via image captions only; their full rendered content cannot be verified from the extracted text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.