Seeing Through Conflicts: Improving Instruction Hierarchy Alignment in Vision-Language Models
Nicholas Sansoterra, Zishuo Zheng, Sachin Kumar
cs.CL · cs.CV · cs.LG
Quick verdict
RL training on multimodal instruction conflicts cuts VLM jailbreak ASR; rigorous, but 'mixed is best' fails on mirrored attacks where prompting alone wins.
Summary
The paper extends instruction hierarchy (IH) alignment — teaching models to prioritize system instructions over user requests — to vision-language models. Building on the text-only VerIH dataset (7,192 examples), the authors render user prompts as typographic images (VerIH-IMAGE) and mix both modalities (VerIH-MIXED), training Qwen3-VL-4B/8B-Thinking with GRPO under rule-based rewards. Across five configurations they evaluate multimodal jailbreaks (FigStep, MML-WR, MML-MR), real-image (MemeSafetyBench) and agentic (SafeArena) safety, plus text-only and general capability. VerIH-mixed cuts the 4B FigStep attack success rate from 27.97% to 0% and MML-WR from 29.0% to 5.4%; on MemeSafetyBench (8B) ASR falls from 15.23% to 0.02% while benign success rises. An ablation shows aligned-only image training fails on cross-modal attacks (MML-WR 22.6% vs 5.4%), so conflict supervision — not mere multimodal exposure — is the active ingredient. Rigor is above average: McNemar tests with Holm correction, Attack@3 resampling, full score distributions, and judge validation (80% accuracy, F1 0.756). However, 'mixed performs best' fails on MML-MR with the 8B model, where prompting alone (10.62%) beats every fine-tuned variant (15.34%–19.25%) under unstable reconstruction; training uses only clean typographic images and a single model family, contains no safety data, and MMLU drops from 82.57 to 79.07. Overall a careful, well-bounded empirical study whose robustness gains are meaningful but partly model-specific.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Text-only IH training transfers only partially to multimodal attacks, failing when instructions must be decoded, reconstructed, or reasoned across modalities.
- RL training with rule-based rewards on image-rendered conflicts reduces multimodal jailbreak ASR substantially — e.g., 4B FigStep 27.97%→0% and MML-WR 29.0%→5.4% with VerIH-mixed.
- Conflict supervision, not mere multimodal exposure, is the active ingredient: aligned-only image training fails on cross-modal attacks (4B MML-WR 22.6% vs 5.4%).
- Gains generalize beyond synthetic typographic training to real-image (MemeSafetyBench 15.23%→0.02%) and web-agent (SafeArena 26.0%→18.0%) safety while largely preserving general capability.
Strengths
Controlled modality isolation
VerIH-IMAGE is a one-to-one transformation of VerIH-TEXT (7,192 examples) in which only the user prompt is re-rendered as a typographic image; system prompt, alignment label, and reward verifier are untouched (§3.2), cleanly isolating modality as the manipulated variable.
Ablation identifies the active ingredient
The aligned-only image variant (§6.3, Table 9) fails on cross-modal attacks — 4B MML-WR ASR 22.6% vs 5.4% for VerIH-mixed, MML-MR 24.2% vs 8.08% — showing conflict supervision, not multimodal exposure, drives the gains.
Above-average statistical hygiene
Generalization beyond the synthetic training distribution
Transparency and honest failure analysis
Weaknesses
The 'mixed-modality performs best overall' claim is contradicted on 8B MML-MR
Judge recall bounds all near-zero ASR headlines
Training-vs-prompting attribution is entangled by design
Narrow scope and weak agentic baseline
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper extends instruction hierarchy (IH) alignment — teaching models to prioritize system instructions over user requests when they conflict — to vision-language models, where conflicting instructions can arrive as text rendered in images, mirrored, or split across modalities. Three central claims: (1) text-only IH training transfers only partially to multimodal attacks; (2) RL with rule-based rewards on image-rendered and mixed-modality conflicts substantially cuts attack success rates (ASR), with mixed supervision claimed 'best overall'; (3) gains generalize to real-image and web-agent safety while preserving general capability. The work builds directly on VerIH (Zheng et al., 2026) and releases code and data.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is instruction hierarchy (IH) alignment?
It is training language models to prioritize higher-priority instructions — such as system prompts — over lower-priority ones (like user requests) when the two conflict; for example, refusing a user request that violates a system-level safety rule.
What does this paper add over prior instruction-hierarchy work?
Prior IH work was text-only. This paper extends IH alignment to vision-language models by rendering user instructions as images (typographic attacks) and training with reinforcement learning on rule-verifiable conflicts, showing that text-only IH training does not fully transfer to multimodal attacks.
Did the mixed-modality trained model win on every benchmark?
No. It was best on most settings (e.g., 4B FigStep 0% ASR, MML-WR, MemeSafetyBench), but on mirrored-text MML-MR with the 8B model, prompting alone reached 10.62% ASR versus 15.34%-19.25% for all fine-tuned variants, and the same inversion appears under Attack@3 resampling.
Is the code and data for this paper available?
Yes — the paper states that code and data are available at github.com/skai-research/Seeing-Through-Conflicts, and training uses public benchmarks (FigStep, MML, MemeSafetyBench, SafeArena, IHEval, MMLU, MMMU, MMBench).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Table 1's header row is garbled in the PDF extraction (column labels interleaved); column values were reassembled from prose descriptions and cross-checked against Attack@3 and Appendix B tables, so specific cell attributions are reliable but were not read from a clean header.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.