PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift
Yusuf Khalid Shire, Sang-Chul Kim
cs.CR · cs.AI
Quick verdict
Proposes a frozen multi-axis PI-detector benchmark showing F1 hides benign over-defense; claims carefully scoped, statistics sound, reproducibility strong.
Summary
Published in IEEE Access, this paper introduces PIDS-Bench, a frozen, checksum-verified benchmark that evaluates binary prompt-injection gateway detectors jointly on attack detection and benign false positives at fixed thresholds, across five conditions: in-distribution, hard-benign, obfuscated, domain-shift, and structural-shift. Seven detectors are compared: three internally trained baselines (TF-IDF+LR, DistilBERT, DeBERTa-v3-FT), two external injection classifiers, two broad-safety comparators, plus a rule-based lower bound. The headline finding: DeBERTa-v3-FT reaches IID F1 0.9882 yet blocks 31.4% of externally-sourced, security-adjacent benign prompts, and across a full threshold sweep and five seeds no internal detector meets F1 ≥ 0.95 with hard-benign FPR ≤ 0.10 (minimum attainable 0.226–0.277). Hard-negative augmentation eliminates over-defense on curated inputs (0.48→~0.00) but not externally-sourced ones (0.31→0.31; 0.46→0.39), a pattern the authors term provenance-sensitive over-defense; instruction-like structure dominates yet source provenance retains a significant independent effect. Analyses rule out calibration artifacts via temperature scaling, and lexical evidence is available only for the linear model. Limitations: English single-turn inputs only, 23.8% of the test split is template-generated, augmentation matched to the externally-sourced distribution remains untested, and stress-set FPRs should not be read as deployment false-positive rates.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A detector with IID F1 0.9882 (DeBERTa-v3-FT) still blocks 31.4% of externally-sourced security-adjacent benign prompts at τ=0.5, showing aggregate F1 conceals over-defense.
- Across a full threshold sweep and five seeds, no internal detector jointly achieves F1 ≥ 0.95 and externally-sourced hard-benign FPR ≤ 0.10; the constrained minimum is 0.226–0.277 (Table 9).
- Hard-negative augmentation nearly eliminates curated over-defense (0.48→~0.00) but leaves externally-sourced over-defense intact (0.31→0.31; 0.46→0.39), across both architectures and pool sizes 100–419 — termed provenance-sensitive over-defense.
- Over-defense is neither a thresholding nor a calibration artifact: temperature scaling halves ECE with unchanged FPR (Table 11), and over-blocked inputs score deep in the injection region (32.1% ≥ 0.9 for seed 42).
- Instruction-like structure is the dominant driver of externally-sourced over-defense, but source provenance retains a statistically independent effect (partial correlations 0.248 vs 0.169, §6.4).
Strengths
Detector-level two-sided protocol fills a real gap
End-to-end benchmarks (BIPIA, InjecAgent, AgentDojo) conflate detector behavior with downstream model compliance (§2.3); PIDS-Bench isolates the binary gateway and treats benign FPR as a primary metric (§5.1), realized through five frozen stress sets (Table 3).
Statistical reporting above venue norm
Five-seed means with 10,000-resample bootstrap CIs (Tables 17–18), a paired three-test augmentation analysis that explicitly handles the Wilcoxon floor at n=5 (0.0625, Table 12), and justified abstention from cross-group significance testing given differing threshold regimes (§4.3).
Mechanism triangulation rather than single-evidence claims
Reproducibility and license handling
Honest claim scoping
Weaknesses
Only one mitigation family tested
Limited external-detector comparability
Stress-set circularity and representativeness
n=5 limits fine-grained inference
Transformer mechanism unproven
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
PIDS-Bench (arXiv:2609.15017; IEEE Access vol. 14, 2026) is a frozen, checksum-verified benchmark for binary prompt-injection gateway detectors. The central claim is that aggregate held-out F1 conceals benign-side over-defense: DeBERTa-v3-FT reaches IID F1 0.9882 ± 0.0008 (Table 6) yet blocks 0.3144 ± 0.0525 of the externally-sourced hard-benign subset at τ=0.5 (Table 8). Secondary claims: no internal detector meets F1 ≥ 0.95 with hard-benign FPR ≤ 0.10 at any swept threshold or seed (Table 9); hard-negative augmentation clears curated over-defense but not externally-sourced over-defense — 'provenance-sensitive over-defense' (Table 10); and the failure is neither a thresholding nor a calibration artifact (§6.2, §6.6). Scope is deliberately narrow: single-turn English, static non-adaptive attacker, detector-evasion success only (§3.5, §8) — a framing the paper states plainly and mostly maintains throughout.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is PIDS-Bench?
PIDS-Bench is a frozen, checksum-verified benchmark for evaluating prompt-injection detectors on both attack detection and false positives on benign, security-adjacent inputs. It spans five conditions — in-distribution, hard-benign, obfuscated attacks, domain shift, and structural shift — with seven detectors compared under a fixed-threshold protocol and all splits released with SHA-256 checksums.
What is provenance-sensitive over-defense?
It is the paper's term for the finding that training a detector with hard-negative benign examples removes false positives on curated stress inputs almost completely (0.48→~0.00 for DeBERTa-v3-FT) but leaves false positives on externally-sourced prompts from public corpora largely unchanged (0.31→0.31). The mitigation thus helps most where it is needed least, and this held across two architectures and all tested augmentation pool sizes (100–419).
Can prompt-injection detectors achieve both high F1 and low false positives on security-related benign text?
On this benchmark, no. Across five training seeds and a complete threshold sweep, none of the three internal detectors reached F1 ≥ 0.95 together with a hard-benign false-positive rate ≤ 0.10; the lowest attainable rate under the F1 constraint was 0.226–0.277. Temperature scaling and threshold tuning did not resolve the trade-off.
Is the PIDS-Bench data and code publicly available?
Yes. Splits, stress sets, the augmentation pool, two-annotator audit labels, and evaluation code are released under a tagged GitHub release with SHA-256 checksums in FREEZE_MANIFEST.txt. Two constituent corpora carry non-commercial licenses, restricting use to research; 664 LMSYS-derived rows ship as fingerprints with a verified rebuild script.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Extraction truncated portions of §2.4, §6.2–6.3, and §7.1–7.2; judgments there rely on surrounding text, table references, and the discussion sections.
- Figures 1–3 are assessed from captions and in-text descriptions only, not the rendered graphics.
- The IEEE Access publication line appears in the page header but the peer-review trail of that venue version cannot be verified from this text.
- Numbers are quoted as extracted; minor OCR/typographic risk on decimal values cannot be fully excluded.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.