Connecting the Dots in Agentic AI Security: A Cross-Dimensional Threat Taxonomy, Evaluation Maturity, and Open Challenges
Heewon Baek, Alsharif Abuadbba, Kristen Moore, Hyoungshick Kim, Surya Nepal
cs.CR · cs.AI
Quick verdict
Cross-dimensional survey of agentic AI security: 66 studies, 4-axis threat map, 22 artifact-backed papers; rigorous and honest, but empirical base is small.
Summary
This survey organizes security threats to LLM-based agentic systems along four linked dimensions — affected system surfaces (S), interaction/trust boundaries (B), violated security properties (P), and empirically examined architectures (A) — denoted T=⟨S,B,P,A⟩. From a structured search of ACM DL, IEEE Xplore, Scopus, Web of Science, Google Scholar, and arXiv (2022–2026) plus snowballing, the authors build a 66-study corpus and derive 32 threat families across functional, interaction, and systemic groups (Section IV, Table II). They then separate what is known from what is empirically demonstrated: 22 artifact-backed red-teaming studies (Table III) and 11 representative benchmarks (Table V) are mapped onto the framework. Coverage concentrates on prompt injection/reasoning, memory poisoning, and tool-mediated attacks; 16 of 22 empirical studies target single-agent systems, and only 7 evaluate a concrete defense; 10 of 11 benchmarks are single-agent. The authors attribute weak comparability to heterogeneous ASR definitions, limited adaptive-defense evaluation, short horizons, and incomplete execution-state capture, and derive 13 open research questions. Main limitations: findings are explicitly scoped to the selected corpus; the artifact criterion may bias the coverage picture; no inter-coder reliability for the threat coding is reported; and no aggregated coverage matrices or quantitative screening counts appear in the provided text. The three-level success distinction (model/agent/system compromise) is a useful methodological contribution.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Existing agentic-AI threat classifications organize threats along single dimensions, obscuring how compromise traverses systems; the four-axis T=⟨S,B,P,A⟩ representation addresses this across 66 studies (2022–2026).
- Within the 22 artifact-backed red-teaming studies, evidence concentrates on prompt/reasoning, memory, and tool-mediated attacks, with 16 of 22 studies in single-agent settings.
- Evaluation maturity is constrained by heterogeneous success metrics, concrete defense evaluation in only 7 of 22 studies, architectural imbalance (10 of 11 benchmarks single-agent), and incomplete execution-state capture.
- Public artifacts enable inspection but do not establish reproducibility, motivating 13 open research questions for standardized, architecture-aware evaluation.
Strengths
Cross-dimensional framework with disciplined epistemic scoping
T=⟨S,B,P,A⟩ (Sec. IV, Table II) separates entry boundary from affected surface and consequence — e.g., Table IV contrasts AgentDojo tool-output injection with AiTM inter-agent message manipulation — and the paper repeatedly scopes coverage claims to the selected corpus rather than the field.
Known-vs-demonstrated threat separation
RQ2/RQ3 are restricted to 22 studies passing four criteria including mandatory public artifacts (Table III, C/P/D/E badges), with 10 of the 11 benchmarks (Table V) drawn from that same corpus; this avoids conflating survey-documented threats with empirically tested ones.
Actionable outcome taxonomy
Current, concretely extracted evidence base
Weaknesses
Small, criterion-biased empirical subset
No coding reliability or screening accounting
Aggregates asserted in prose, not tabulated
Benchmark representativeness and the M label under-specified
No independent artifact verification
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This survey (arXiv 2609.23894, cs.CR/cs.AI) attacks a real structural problem: threat taxonomies organized along one axis (component, mechanism, lifecycle, or objective) hide how compromise moves through a system — content retrieved via a tool (boundary) can poison reasoning (surface), persist in memory (surface), and later trigger an unauthorized action (property). The authors propose T = ⟨S, B, P, A⟩, cross-mapping 32 threat families (Table II) over surfaces, trust boundaries, security properties, and empirically examined architectures, derived from a 66-study corpus (2022–2026). Three research questions structure the paper: RQ1 (characterize the attack surface), RQ2 (what has been empirically tested), RQ3 (evaluation maturity). Central empirical claims: coverage concentration on prompt/reasoning, memory, and tool-mediated attacks; 16/22 single-agent studies; only 7/22 defense evaluations; 10/11 single-agent benchmarks; heterogeneous ASR semantics; incomplete execution-state capture. Thirteen open questions (OQ1–OQ13) operationalize the gaps. Unusually for this genre, the paper repeatedly hedges: findings 'describe the selected corpus rather than establish gaps across all empirical research.'
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the T=⟨S,B,P,A⟩ framework in this paper?
It is a four-part label the authors attach to each agentic-AI threat: the affected system surface (S, e.g., reasoning, memory, tools), the trust boundary where compromise enters or propagates (B, e.g., agent–tool, agent–agent), the security property violated (P, e.g., goal integrity, confidentiality), and the architectures where the threat has been empirically evaluated (A, e.g., single-agent, hierarchical). Unlike single-axis taxonomies, it links where an attack manifests, how it propagates, and what it breaks.
What are the main findings of the survey?
Across its 66-study corpus (2022–2026), documented threats are broad, but among the 22 red-teaming studies that released public artifacts, tested attacks concentrate on prompt injection/reasoning, memory poisoning, and tool-mediated attacks, mostly against single-agent systems (16 of 22). Only 7 of 22 studies evaluated a concrete defense, and 10 of 11 reviewed benchmarks target single agents. Evaluation maturity is limited by inconsistent success metrics, few adaptive-defense tests, short time horizons, and weak execution-state capture.
Does the paper claim agent security research is missing whole areas?
No — it explicitly states its coverage findings describe the selected corpus, not the entire field. It identifies under-covered regions within its artifact-backed subset (persistent/cross-session compromise, Human–Agent oversight, complex multi-agent and systemic threats) and converts them into 13 open research questions.
Is there code or data released with this survey?
The provided text does not announce a released dataset or codebase for the review itself. The paper instead catalogs the public artifacts (code, prompts, datasets, evaluation scripts) of the 22 reviewed studies and 11 benchmarks, linked per row in Tables III and V.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Extracted text contains omitted sections (parts of Sections II-F, V-B, VI and later references); the evaluation-maturity rubric and some benchmark discussion could not be assessed.
- Figures 1–4 and Tables I–V are only partially captured as text; coverage conclusions could not be verified against the full tables.
- No aggregated per-dimension coverage counts appear in the provided text; 'concentration' claims rest on prose plus stated counts.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.