Prefilling the Reasoning Channel: Output-Prefix Attacks on Reasoning LLMs
Lukáš Brůna, Robert Bridges, Adam Ek
cs.CR · cs.AI
Quick verdict
Reasoning-prefill turns inert injection into up-to-99% ASR jailbreaks; clean 3x2 factorial on 3 models, but no significance tests, thin judge validation.
Summary
This paper reports the first controlled study of output-prefix ('assistant prefill') attacks targeting the scratchpad reasoning channel of reasoning LLMs. In a 3x2 factorial design (prefix: none/static/contextual x reasoning injection: absent/injected), 1,800 test cases (100 AdvBench prompts x 6 tactics x 3 models) probe Gemini 3 Flash Preview, DeepSeek V4 Flash, and Claude Haiku 4.5; an LLM judge scores completions on a three-point scale, with attack success rate (ASR) defined as the Hit rate. Malicious reasoning injected alone is essentially inert (~0% ASR), yet the same reasoning plus a trivial unfinished output prefix raises ASR to 99% on Gemini and 76% on DeepSeek; contextual prefixes beat static ones; Claude Haiku 4.5 remains at <=1% throughout, showing the weakness is fixable. The authors offer a speculative token-distribution account of the null reasoning-only effect and map which APIs still accept prefill. Limitations: three models and one dataset only; a single judge model (itself a victim) manually validated on just one cell; n=100 per cell without confidence intervals or significance tests; the exposed- vs hidden-reasoning comparison is confounded by model identity; the mechanistic hypothesis is untested. Code is public.
Review complete
Assessment at a glance
0
risk flags identified
● high 0
● medium 0
● low 0
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Injecting malicious scratchpad reasoning alone is essentially inert (~0% ASR) across all three tested reasoning models.
- The same malicious reasoning combined with a trivial unfinished output prefix raises ASR to as high as 99% (Gemini 3 Flash Preview) and 76% (DeepSeek V4 Flash).
- Contextual (dataset-pair) prefixes outperform static prefixes, and the reasoning vector works even on hidden-reasoning models via a fabricated <think> block.
- Susceptibility is strongly model-dependent; Claude Haiku 4.5 stays at <=1% ASR under every tactic, showing the vulnerability is addressable at the harness level.
Strengths
Genuinely controlled factorial design
The 3x2 design (Table 1) cleanly separates reasoning-only, prefix-only, and combined effects over the same 100 AdvBench prompts per model - the first controlled isolation of the reasoning channel per Sec. 3.
Actionable null result with qualitative corroboration
Reasoning-only injection yields ~0% ASR everywhere (Table 3, tactic 4), corroborated by the matched transcript pair in Appendix F where the identical scratchpad refuses without a prefix and complies with one.
Practitioner-relevant harness documentation
Transparency: judge validation, prompts, disclosure
Weaknesses
No statistical inference
Judge circularity
Harness factor confounded with model identity
Single payload generator
Mechanistic account untested
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper claims the first systematic, controlled study isolating the scratchpad reasoning channel as an output-prefix attack vector, motivated by three research questions (Sec. 1): R1 (are reasoning LLMs vulnerable to prefix attacks), R2 (does attacking harness-exposed or fabricated scratchpad reasoning help), and R3 (mitigations). Two hypotheses are tested: H1 (exposed-reasoning models are more robust to prefix-only edits) and H2 (scratchpad tokens are an additional attack vector). Headline claims: reasoning-only injection is inert (~0% ASR); the same reasoning plus a trivial unfinished prefix reaches 99% ASR; contextual prefixes beat static ones; Claude Haiku 4.5 is near-immune. The novelty positioning against ChatBug [13], OPRA [28], and ChatInject [6] (Sec. 3) is argued fairly - ChatInject's user-turn <think> injection is acknowledged as closest, with differentiation lying in harness-level reasoning_content injection and the controlled factorial.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 0 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is an output-prefix (prefill) attack on an LLM?
An attacker with API access supplies text in the assistant's response field, so the model continues generating from that planted text instead of starting fresh. Because LLMs are stateless and auto-regressive, they treat the planted 'previous output' as their own, letting the attacker steer the answer toward prohibited content almost for free - no gradients or special compute required.
Does injecting malicious text into a reasoning model's thinking channel alone jailbreak it?
Not in this study. Across 1,800 test cases on Gemini 3 Flash Preview, DeepSeek V4 Flash, and Claude Haiku 4.5, injecting a malicious reasoning scratchpad alone produced almost no successful attacks (about 0%). The same scratchpad becomes dangerous only when paired with a short, unfinished output prefix such as 'Sure, I would love to help with that, here is', pushing attack success up to 99% on the most vulnerable model.
Which models resisted the prefill attack?
Claude Haiku 4.5 stayed near 1% attack success in every condition, while Gemini 3 Flash Preview reached up to 99% and DeepSeek V4 Flash up to 76%. The authors argue this shows the vulnerability is a serving-harness problem vendors can fix - Anthropic reportedly lists prefill susceptibility as an evaluation criterion - rather than an unavoidable flaw of reasoning models.
Can providers fix this attack?
Largely yes: endpoints can ignore client-supplied assistant turns (OpenAI's endpoints do), drop prefill support, or screen supplied reasoning content. The paper notes a caveat: Gemini-3's native endpoint removed prefill, but its OpenAI-compatibility endpoint still accepts it, so closing one door is insufficient while compatibility layers keep another open.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 (stacked bars, ASR heatmap, CoT diagram) cannot be visually analyzed from extracted text; all numeric claims were verified from Tables 3-5 instead.
- Extraction appears complete (abstract through Appendix I), but minor formatting in Listings 1-3 was verified only as text.
- Table 4's endpoint behavior is explicitly time-sensitive ('facts as of our evaluation', 2026) and could not be independently re-verified here.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.