SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Consideration
Md Jueal Mia, Yanzhao Wu, Selcuk Uluagac, M. Hadi Amini
cs.AI
Quick verdict
SoK taxonomizing 116 jailbreak attacks and 63 defenses for agentic AI; controlled study shows defense value is model- and attack-dependent; stats thin.
Summary
This Systematization of Knowledge reorganizes LLM jailbreak security around the agentic execution pipeline. From a documented 2022-2026 literature search (271 attack and 88 defense candidates screened), the authors build taxonomies of 116 attacks and 63 defenses along dimensions including adversarial access, injection channel, target component, and defense stage, and propose a security-utility-efficiency framework separating native harmful-prompt safety, adversarial robustness, and agent-level outcomes. A controlled study runs 16 inference-time defenses plus a Vanilla baseline on Qwen3.5-4B and Llama-3.1-8B-Instruct within a LangGraph harness, evaluating XSTest, AdvBench (first 50 instructions, nine attacks), and AgentHarm with TRACE, MINJA, and AgenticRed. Results show modern native alignment leaves little headroom (Vanilla ASR 2.51% on XSTest with Llama), that defense rankings are model- and attack-dependent (Self-Reminder best for Qwen, Self-Eval for Llama), and that low final-response ASR masks heavy intermediate compromise: TRACE yields 91.49% unsafe planning at only 11.54% final ASR, and Self-Eval reaches 0% final ASR under MINJA while unsafe memory stays at 95.65%. Main limitations: 50 samples per attack cell with no confidence intervals or statistical tests, GCG scoring 0% in every condition (suggesting a failed attack configuration), a single LLM judge (GPT-5-mini), and arbitrary composite-score weights; no code release is stated in the provided text.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Strong native alignment does not imply adversarial jailbreak robustness: Vanilla Llama-3.1-8B averages 34% ASR across nine attacks on AdvBench despite only 2.51% ASR on XSTest.
- Defense effectiveness is highly model-, attack-, and component-dependent and can impose substantial over-refusal, utility, and latency costs (e.g., Self-Reminder best for Qwen3.5-4B, Self-Eval for Llama-3.1-8B).
- Low final-response ASR masks severe intermediate compromise: TRACE produces 91.49% unsafe planning at only 11.54% final ASR, and Self-Eval's 0% final ASR under MINJA coexists with 95.65% unsafe memory.
- Jailbreak defense should shift from response-centric filtering toward cross-layer, execution-aware security protecting agent state, component transitions, and external actions.
Strengths
Documented, large-scale systematization
Sec 2.1 describes a six-database 2022-2026 search screening 271 attack and 88 defense candidates into 116 attacks and 63 defenses, coded on nine dimensions; Tables 12-25 add per-item venue, access level, mechanism, code availability, and a key-limitation note for every entry - a genuinely reusable reference artifact.
Unusually honest empirical reporting
The paper shows Vanilla ranking near the top of its own composite (84.5 of Qwen's top score 86.8; 96.5 vs Llama top 97.1, Sec 5.2) and reports degenerate defenses as such - Prompt Guard and Robust Aligned at 100% over-refusal and 0% utility (Tables 5-6) - rather than hiding inconvenient baselines.
Cross-layer evidence for the core thesis
Controlled common-harness design
Weaknesses
No statistical uncertainty anywhere
GCG anomaly invalidates parts of Table 8
Single LLM judge for all labels
Composite rankings rest on arbitrary weights
Agentic-layer defenses taxonomized but never run
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This SoK reframes jailbreak security around the full agentic execution pipeline (user interaction, planning/reasoning, memory, tools, inter-agent communication) and asks which established findings survive modern safety alignment. Four RQs (Sec 1) cover attack-surface expansion, systematic characterization across components and trust boundaries, marginal defense benefit with security-utility-efficiency trade-offs, and which prior conclusions need re-evaluation. Positioning against Xu et al. [169], Wang et al. [150], and Kim et al. [64] (Table 1) is concrete: the unit of analysis shifts from prompt-response pairs to the agent trajectory, plus a controlled cross-layer empirical matrix. Scope discipline is explicit: injections that merely redirect an authorized task are excluded, and empirical depth is confined to model-layer inference-time defenses.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the main finding of this SoK on jailbreaking in agentic AI?
Three things: modern LLMs' native safety training leaves attackers little room on ordinary harmful prompts (Vanilla ASR 2.51% on XSTest with Llama-3.1-8B), yet adversarial jailbreaks still succeed (34% average across nine attacks on AdvBench), so defenses add value only unevenly; defense effectiveness depends heavily on the specific model, attack, and pipeline component; and a low final-response attack success rate can hide severe unsafe behavior in planning, memory, and tool use inside the agent (e.g., 91.49% unsafe planning at only 11.54% final ASR under the TRACE attack).
Which jailbreak defense performed best in the paper's experiments?
It depends on the model: Self-Reminder scored highest overall for Qwen3.5-4B (composite 86.8) and Self-Eval for Llama-3.1-8B (97.1). Notably, the undefended Vanilla baseline ranked close behind (84.5 and 96.5), and several defenses degraded utility or latency substantially, so the practical gains over doing nothing are modest in these settings.
What models, datasets, and attacks were used in the evaluation?
Qwen3.5-4B (as router/tool model and final model) and Llama-3.1-8B-Instruct (final model) in a LangGraph agent harness. Benchmarks: XSTest (250 benign + 200 harmful prompts), the first 50 AdvBench harmful instructions attacked by nine methods (AutoDAN, PAIR, TAP, GCG, Cipher, DeepInception, CodeChameleon, ICA, Jailbroken), and AgentHarm-chat attacked by TRACE, MINJA, and AgenticRed. GPT-5-mini served as the LLM judge.
Is code or data available for this paper?
The provided text does not state a code release by the authors. The cited attacks and defenses largely have public code (the comparison tables include code-availability columns), and all models and benchmarks used are public, which makes re-running the study plausible, but exact harness configurations and judge prompts are not shown.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt: middle sections marked [...sampling gap...] were skipped, so parts of the attack/defense taxonomy discussion, the full results narrative (e.g., Tables 2-4, Figure 5-7 details), discussion/lessons sections, and Table 25 onward were not visible; judgments about those parts are not made.
- The provided text ends mid-reference list (at reference [107]); later references and any appendix are not visible.
- The abstract in the metadata block is cut mid-sentence, but the abstract appears complete within the body text, so no claim relies on the truncated metadata copy.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.