Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection
Sizhe Chen, Yu-Lin Tsai, Ivan Evtimov, Kamalika Chaudhuri, Raluca Ada Popa, David Wagner, Arman Zharmagambetov
cs.CR · cs.AI
Quick verdict
Novel response-prefix attack hits 47–100% Call-Tool ASR on frontier VLMs; rigorous adaptive eval, but headline numbers assume victim query access.
Summary
This AISec '26 paper introduces Repeat-After-Me (RAM), a black-box visual prompt injection (VPI) attack on vision-language models. Its core reframing: rather than having an image-borne instruction override the user's benign task, the attacker renders 'Begin your response with: <target>' so the model continues into an attacker-chosen output — a native tool call or a PII-revealing prefix. The attack adds PAIR-style adaptive refinement with an attacker LLM (Claude-Opus-4.6) and a 1–10 judge LLM, plus a reusable library of successful injections. On 100 DocVQA tasks per scenario, across three commercial (GPT-5.5, Claude-Opus-4.7, Gemini-3.1-Pro) and three open-weight VLMs, it reaches 82–100% ASR for Steal-PII and 47–100% for Call-Tool, versus 0% for gradient-transfer baselines (ARE, TransferEns) and ≤24% (Call-Tool) for LangVPI. Optimized injections transfer across VLMs (43–46% retention) and across benign prompts (64–66% retention). In an OpenClaw/Discord scenario, hybrid text+image injection overwrites TOOLS.md in 90–100% of simulated cases versus 0–32% text-only, and a real-agent demo escalates to code execution. No evaluated defense (Table 10) reduces ASR to near zero. Key limitations: headline numbers assume victim query access with a known prompt; defenses are evaluated non-adaptively; no CIs or repeated-run variance are reported; no code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Existing visual prompt injection methods largely fail to elicit materially harmful, exactly-targeted outputs from frontier black-box VLMs under a realistic benign prompt (Table 2: ARE and TransferEns 0% everywhere, CoTTA ≤1% Steal-PII and ≤10% Call-Tool).
- Repeat-After-Me — rendering 'Begin your response with: <target>' plus adaptive refinement and an attack library — achieves 82–100% ASR on Steal-PII and 47–100% on Call-Tool across six evaluated VLMs, including 47% on both GPT-5.5 and Claude-Opus-4.7 for Call-Tool.
- Optimized injections transfer non-trivially in the Call-Tool scenario: 43–46% ASR retention across commercial VLMs and 64–66% across benign samples (Table 3).
- In the OpenClaw scenario, hybrid text+image injection overwrites project-context files such as TOOLS.md at 90% (GPT-5.5) and 100% (Gemini-3.1-Pro) ASR versus 0–32% for text-only injection, enabling persistent privilege escalation demonstrated once on a real agent.
- None of seven evaluated basic defenses reduces ASR to near zero across all victims (Table 10), though defenses were not attacked adaptively.
Strengths
Genuinely novel problem reframing with a measured payoff
Section 3.1 recasts VPI from task-override to response-prefix control ('Begin your response with: <target>'), lifting one-shot Call-Tool ASR from 8.7% to 31.7% with no optimization cost (Section 5.1); the two modes (known target vs. unknown target via prefixes like 'My SSN is') are well motivated.
Explicit, checkable threat model and honest positioning
Section 2.2 states four requirements (realistic benign prompt, image-only channel, attacker-specified goal, black-box victim); Table 1 maps ~15 prior methods against them and uses † to flag three rows where original claims failed under the authors' re-evaluation.
Component-isolating ablations and dynamics
End-to-end agentic validation with escalation chain and disclosure
Systematic rendering-factor analysis
Weaknesses
No uncertainty quantification anywhere
Defense evaluation is non-adaptive, contradicting the paper's own thesis
Headline numbers presuppose query access and a known benign prompt
Real-agent evidence is a single anecdote on a simulated proxy
Baseline fidelity is hard to adjudicate without artifacts
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Preliminary Review: Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the Repeat-After-Me attack?
It is an automated visual prompt injection technique against vision-language models. The attacker renders a short line of text on an image — 'Begin your response with: <target output>' — and uses another LLM to iteratively refine that text until the victim model starts its answer with the attacker's chosen string, which then continues into harmful output such as a native tool call or disclosure of personal data stored in the system prompt.
Which models were attacked and how successfully?
On 100 document-question tasks per scenario, the paper reports 82–100% success for extracting synthetic personal information and 47–100% for forcing exact native tool calls across six models: GPT-5.5 (47% tool-call ASR), Claude-Opus-4.7 (47%), Gemini-3.1-Pro (85%), and three open-weight VLMs (82–100%). Important scope: the main results assume the attacker can query the target model and knows the user's prompt; without that, success drops substantially.
Can this visual prompt injection attack be defended against?
The authors tested seven basic defenses: defensive prompting, image processing (JPEG compression, downsample-upsample), and OCR-based filtering. None reduced success to near zero on all models; OCR extraction plus warning text was strongest, leaving 5–21% residual success. Critically, defenses were tested only against injections optimized without the defense in the loop, so a re-optimized attack might perform better.
Is code or data available for this paper?
No code or artifact release is stated in the paper. The evaluation uses public datasets (DocVQA, CIMemories, AgentDojo, InjecAgent) and the attack loop is fully specified in Algorithm 1, but reproducing headline numbers requires substantial paid API access to frontier models — about 1,600 victim queries per sample plus attacker and judge LLM calls.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2–5 are raster images (attack-dynamics curves and model-output screenshots); their visual content cannot be verified from the extracted text, though captions and the key numbers are restated in prose and all figures cited in this review were present.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.