ENDOPROMPT: Victim-Side Pseudo-References for Utility Degradation
Qingyu Wu, Zeyu Feng, Yongda Yu, Yuzhe Luo, Hua Cheng
cs.AI · cs.CR
Quick verdict
ENDOPROMPT learns utility-degrading prefixes from unlabeled victim signals (-26.8pp, 27/28 cells); honest, but its request-matching advantage is unproven.
Summary
ENDOPROMPT is a white-box prompt-injection method that trains a generator to produce short prefixes degrading an LLM's benign task performance, without task labels or prescribed failure responses. The core idea is to use the victim's own clean continuation on unlabeled instructions as a pseudo-reference: prefixes are ranked by how much they raise the NLL of this fixed continuation, and this displacement reward drives local search, within-instruction pairwise preference fitting (β=0.1), and GRPO refinement of a request-conditioned generator initialized from Dolphin3.0-Llama3.2-3B. Training uses 2,400 Dolly-15k instructions; benchmark data are held out. Across four instruction-tuned victims (Qwen2.5-7B, Llama-3.1-8B, Mistral-7B-v0.3, Gemma-2-9B) and the complete splits of seven benchmarks, the mean utility change is -26.8 pp (95% CI [-27.4, -26.2]), with 27 of 28 cells negative. Failure analysis attributes regressions mainly to output expansion (58.4%) and answer-extraction failure (39.5%), rarely refusal (0.3%). Notably, the authors report that fixed-generator controls fail to establish any advantage from request matching (matched-minus-shuffled +0.21 pp), and prefix reuse is heavy (distinct ratios 0.4-4.7%). Main limitations: a single seed per victim, unmatched reference attacks, only a 3.2 pp net gain over the pre-fit generator, and code not yet released.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Clean victim continuations on unlabeled instructions can supervise utility-degrading prefixes without task labels, benchmark feedback, or prescribed failure responses (Eq. 2 displacement reward).
- ENDOPROMPT amortizes victim-side local search into a generator that produces a ≤16-token prefix per request at deployment, achieving a mean utility change of -26.8 pp across 28 victim-benchmark cells (27 negative).
- Fixed-generator controls do not establish any degradation advantage from request matching, and the generator often reuses a small set of prefixes (distinct ratios 0.4-4.7%).
- Utility regressions are dominated by fluent output expansion (58.4%) and answer-extraction failure (39.5%) rather than refusal (0.3%), so benign utility must be measured directly, not inferred from refusal rates.
Strengths
Genuinely label-free supervision signal
The displacement reward in Eq. (2) (§3.2) requires only the victim's greedy ≤64-token clean continuation as a pseudo-reference — no benchmark answers, no target failure strings, unlike GCG/AdvPrompter/UAT. This cleanly separates attack discovery on Dolly-15k instructions from downstream evaluation.
Unusual candor in negative controls
§4.4 reports that fixed-generator controls fail to establish request-matching advantage (matched-minus-shuffled +0.21 pp, CI [-0.06,+0.48]; matched-minus-empty -0.30 pp, CI [-0.65,+0.06]) and that the modal prefix increases degradation by 2.79 pp. Self-refuting evidence of this kind is rare and strengthens trust in the rest of the reporting.
Careful evaluation protocol
Informative failure analysis
Weaknesses
Central 'request-conditioned' premise is causally unsupported
Full pipeline buys only 3.2 pp over the pre-fit generator, from a single seed
Reference attacks are not comparable
No multiple-comparison handling and no seed variance
Reproducibility deferred and an unexplained provenance note
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ENDOPROMPT (arXiv 2609.29948) addresses a threat distinct from jailbreaking: prompt-injection prefixes that degrade benign utility — a response can be fluent yet abandon the task (Fig. 1's GSM8K example). The paper claims that (1) a victim's own clean continuations can supervise utility-degrading prefixes without task labels or prescribed failure responses; (2) victim-side search can be amortized into a one-pass, request-conditioned generator; and (3) this yields a mean utility change of -26.8 pp across four victims and seven benchmarks, with 27 of 28 cells negative. The abstract and §4.4 also candidly concede that the controls do not establish a request-matching advantage — a claim-evidence gap the authors flag themselves, which shapes the whole review.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ENDOPROMPT and what does it attack?
ENDOPROMPT is a white-box attack method that trains a small generator to output a short text prefix which, when prepended to a user's request, makes an instruction-tuned LLM perform worse on its benign task (wrong, missing, or rambling answers) — without trying to elicit harmful content, without task labels, and without a predefined failure response.
How much does ENDOPROMPT degrade model performance?
On four instruction-tuned models (Qwen2.5-7B, Llama-3.1-8B, Mistral-7B-v0.3, Gemma-2-9B) and the full test splits of seven benchmarks (GSM8K, MATH-500, BBH, HellaSwag, TruthfulQA, IFEval, MMLU), the average score drop is 26.8 percentage points, with 27 of 28 model-benchmark combinations showing degradation. The only exception is Mistral on MMLU (+1.8).
Does the attack generate a different prefix for each request?
Not effectively. After normalization, only 0.4%-4.7% of generated prefixes are distinct, and a single prefix accounts for up to 65% of requests on one model (Mistral). The authors' own controls found no evidence that matching the prefix to the request strengthens the attack.
Is the code or the attack strings available?
The paper states code will be released upon acceptance, and attack artifacts are shared only with vetted researchers under responsible disclosure. As of the paper's release, neither is publicly available.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are available only as embedded numbers and captions in the extracted text; visual details (e.g., Fig. 3 reward trajectory shapes) cannot be independently verified.
- The metadata author list omits Renju Liu, who appears in the paper byline; metadata and text disagree slightly.
- No appendix/supplementary material is included, so hyperparameter provenance (β=0.1, 0.03 margin, thresholds) cannot be cross-checked.
- Single-seed reporting in the paper itself limits any assessment of training variance, independent of extraction quality.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.