Decoding Guardrails: XAI-Guided Perturbation Analysis of Prompt Injection Detection
Fernando Outeda, Gustavo Betarte, Gustavo Betarte, Juan Diego Campo, Fiorella Cravero
cs.CR · cs.AI
Quick verdict
Exploratory XAI case study: Prompt Guard 2 leans on distributed lexical cues; synonym/paraphrase flips it, but single-prompt scope limits claims.
Summary
This exploratory case study applies two local attribution methods, Vanilla Gradient and SHAP, to Prompt Guard 2, Meta's 86M DeBERTa-based prompt-injection guardrail, to characterize its decision logic and test whether explanations enable evasion. On a single DAN-style English target prompt, both methods attribute importance diffusely (SD 0.118/0.169) and agree only on the top token ('Ignore'; Jaccard 0.286 over top-18, RBO 0.439). Saliency-guided token removal needs 21–27 word deletions to flip the label versus >50 random, and the degraded prompt no longer jailbreaks Llama 3.1 8B. Greedy synonym substitution guided by classification confidence flips the guardrail after 12 replacements (24% of text), yielding a jailbreak of comparable utility to the original by an LLM-judged rubric; sentence-level paraphrasing flips it after 9 of 11 clauses. A dataset-scale analysis of 200 prompts per class from xTRam1/safe-guard-prompt-injection finds 105 undetected injections that lack the lexical markers (e.g., 'ignore', 'previous instructions') driving detections. A Spanish replication needs fewer edits (7 synonym substitutions) but uses a different, shorter prompt. Main limitations: core experiments rest on one prompt; no statistical testing; jailbreak utility scored by an LLM on four tasks; no code release.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 4
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Prompt Guard 2's classification relies on the cumulative contribution of many tokens (importance spread SD 0.118 for Vanilla Gradient, 0.169 for SHAP) rather than a few dominant triggers.
- Confidence-guided synonym substitution (12 edits, 24% of the text) and sentence-level paraphrasing (9 of 11 clauses) flip the guardrail's label and can yield working jailbreaks of Llama 3.1 8B.
- Saliency-guided token removal is roughly twice as efficient as random removal (21–27 vs >50 words), but the resulting degenerate prompts no longer jailbreak the underlying LLM.
- Undetected injection prompts (105 of 200 sampled) systematically lack the instruction-override lexical markers (e.g., 'ignore', 'previous instructions') that drive correct detections.
- Explanation methods intended for transparency can simultaneously lower the cost of constructing adversarial bypasses.
Strengths
Dual-explainer design with quantitative agreement metrics
Section 4 quantifies VG/SHAP agreement via Jaccard (0.2857 over top-18), rank-biased overlap (0.4394, p=0.9) and importance spread (SD 0.118 VG / 0.169 SHAP), rather than eyeballing saliency maps; subtoken attributions are explicitly aggregated to word level.
Honest negative results guard against overclaiming XAI's attack utility
The saliency-minimizing selection strategies FAILED in both §4.2 (16 substitutions, confidence stayed >0.9990) and §4.3, and token-elimination prompts stopped jailbreaking Llama 3.1 8B after 16 removals — the gap between saliency and confidence as an attack signal is reported, not hidden.
Guardrail bypass is separated from actual jailbreak
Triangulation across scales and languages
Weaknesses
Core experiments rest on a single prompt
No statistical treatment anywhere
Cross-lingual comparison is confounded
LLM-only jailbreak judging on four tasks
No artifact release
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is an exploratory case study (the authors' own framing, §2.3) of Prompt Guard 2, Meta's 86M-parameter DeBERTa-v2/mDeBERTa classifier used as a prompt-injection/jailbreak guardrail. Using two local attribution methods — Vanilla Gradient (VG) and SHAP — it asks whether the classifier relies on robust signals or sparse lexical triggers (RQ1–RQ5, §1) and whether explanations are actionable for attackers. Central claims: (1) decisions rest on many tokens with modest contributions, not a dominant trigger; (2) confidence-guided synonym substitution (12 edits, 24% of text) and sentence-level paraphrasing (9 of 11 clauses) flip the label and can yield real jailbreaks of Llama 3.1 8B; (3) saliency-guided token removal beats random removal roughly 2:1 (21–27 vs >50 words); (4) 105 of 200 sampled injection prompts go undetected and lack the instruction-override vocabulary driving detections; (5) explanation methods thus have a dual use. The paper explicitly disclaims building a state-of-the-art attack (§2.3), which the results bear out.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Prompt Guard 2 and why does this paper study it?
Prompt Guard 2 is Meta's open, 86M-parameter classifier (built on DeBERTa-v2/mDeBERTa) that labels input text as benign or as a prompt-injection/jailbreak attempt. It is widely used as a cheap pre-filter in front of LLMs. The paper probes how it actually makes decisions, because its internal logic is opaque to both defenders and attackers.
Can explainable AI tools like SHAP be used to attack LLM safety filters?
Partially, according to this paper. Saliency rankings helped attackers pick targets: saliency-guided word removal flipped the classifier after 21–27 deletions versus more than 50 random deletions. However, raw saliency was a poor guide for choosing edits — selecting synonyms by the classifier's confidence worked better (12 changes, 24% of the text, versus failure at 16 saliency-minimizing changes).
Did bypassing Prompt Guard 2 actually produce working jailbreaks?
Not always. The synonym-substitution bypass produced jailbreaks of similar usefulness to the original DAN prompt (judged by another LLM on four malicious tasks, e.g., functional phishing email), while the sentence-level paraphrase bypass mostly yielded non-functional attacks — only the phishing task remained functional. Bypassing the guardrail label does not guarantee a working attack.
How often does Prompt Guard 2 miss prompt injection attacks?
In this paper's random sample of 200 injection prompts from the public xTRam1/safe-guard-prompt-injection dataset, 105 (52.5%) went undetected. Manual inspection suggested the missed prompts skew toward information-exfiltration phrasing and lack the explicit markers ('ignore', 'previous instructions') that the classifier's detections rely on.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2–5 (saliency bar charts and aggregate word rankings) are referenced but their image content cannot be inspected from the extracted text.
- Minor extraction artifacts exist (e.g., merged running header on page 1); section numbering and all tables appear complete.
- No appendix/supplementary material is available to verify sampling procedure, dictionaries, or judge prompts.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.