ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents
Bingzheng Wang, Xiaoyan Gu, Wentao Wang, Xingyou Yang, Hongcheng Li, Rong Yin
cs.CR · cs.AI
Quick verdict
Pre-execution action auditing via a step-wise tool prior against IPI; strong reported trade-off, but small benchmarks and prompt-scaffold confounds remain.
Summary
This method paper proposes ActGuard, a pre-execution defense against indirect prompt injection (IPI) in tool-using LLM agents. Rather than judging whether external content is suspicious, ActGuard audits each candidate action: a step-wise planner jointly emits the action and a predicted set of plausible next-step tools, forming a soft local tool prior. If the candidate tool falls outside this prior, contrastive log-probability attribution (using a local Llama-3.1-8B proxy) ranks history chunks by how strongly they induce the deviation; if the tool matches, retrieval over explicit argument anchors localizes potentially contaminated parameters. An LLM verifier (GPT-5-mini by default) inspects localized chunks plus neighbors, masks only confirmed malicious spans, and the action is regenerated and re-audited. On AgentDyn and AgentDojo, with trends consistent across GPT-4o-mini, Gemini-2.5-Flash, and DeepSeek-V4-Flash backends, the paper reports pooled average ASR of 2.14% (AgentDyn) and 0.42% (AgentDojo) under GPT-4o-mini, with under-attack utility close to attack-free levels, beating most baselines (DRIFT remains comparable on AgentDojo). Limitations: headline AgentDyn results partly shown only as bar charts; small benchmark scale; no variance or significance testing; no quantitative comparison with the closest concurrent methods; and the audit prompt embeds security rules whose scaffold effect on utility is not isolated.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 3
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Auditing each candidate action against a state-adaptive, soft step-wise tool prior distinguishes legitimate replanning from injection-induced deviations without constraining the execution trajectory (Secs. 1-3).
- Contrastive log-probability attribution against the most plausible prior tool localizes action-inducing external spans better than naive attribution across five proxy models (Fig. 4a).
- Combining tool-level and argument-level localization with verifier-guided masking achieves ASR comparable to state-of-the-art defenses while keeping utility near no-attack levels (Figs. 2-3; Tables 6-9).
- ActGuard resists repeated-payload and verifier-directed adaptive attacks without ASR increase (Fig. 5; Table 10).
- Argument-level localization is the single most critical component: removing it raises ASR from 3.0% to 24.5% on AgentDyn DailyLife (Table 4).
Strengths
Well-motivated reframing of the defense objective
Secs. 1 and 3 shift the question from 'is this content suspicious?' to 'does this content induce the imminent action to deviate from a locally plausible expectation?' The soft tool prior (Eq. 7) is explicitly not an execution whitelist, which distinguishes the design from rigid trajectory constraints and directly targets the replanning-vs-hijacking ambiguity.
Dual-granularity localization is technically sound
Tool-level contrastive attribution (Eq. 9) compares the masked-chunk change in log-probability margin between the candidate tool and the length-normalized best prior tool (Eq. 8, Llama-3.1-8B proxy); parameter anchoring covers attacks that keep a legitimate tool but swap URLs/addresses/commands. Table 4 confirms the argument-level path is the most critical component (ASR 3.0 to 24.5 on DailyLife when removed).
Unusual implementation transparency
Broad, cost-aware evaluation
Utility preservation is genuinely strong where reported numerically
Weaknesses
Prompt-scaffold confound is not isolated
Prior poisoning is unaddressed as an adaptive attack
No inferential statistics or repeated runs
Missing quantitative comparison with the nearest concurrent methods
Verifier design and adaptive-attack coverage show benchmark-adjacent tuning
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ActGuard (arXiv 2609.14987) defends tool-using LLM agents against indirect prompt injection (IPI) under the standard threat model: the attacker controls external content (webpages, emails, tool outputs) but not the user request, system prompt, policy, or tools (Sec. 2, Eqs. 4-6). The core reframing is to audit whether external content *induces the imminent action to deviate from a locally plausible expectation*, rather than whether content is suspicious. Three components (Sec. 3): step-wise planning emitting the action plus a soft predicted next-tool set (Eq. 7); dual-granularity localization (contrastive tool attribution, Eq. 9; parameter anchoring); verifier-guided masking and regeneration. Central claims: ASR comparable to SOTA defenses with utility near the no-attack setting; contrastive beats naive attribution; robustness to two adaptive attacks.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ActGuard and how does it defend LLM agents against indirect prompt injection?
ActGuard is a pre-execution auditing framework: before an agent executes a tool call, it checks the action against a predicted set of plausible next-step tools. If the tool deviates from that expectation, contrastive log-probability analysis finds which past web/email/tool-output text caused the deviation; if the tool looks normal, it traces suspicious argument values (URLs, addresses) back to their source text. An LLM verifier confirms whether the located text is an injection, only confirmed text is masked, and the action is regenerated from the cleaned context before execution.
How well does ActGuard perform compared to other prompt injection defenses?
On the AgentDyn benchmark with GPT-4o-mini, the paper reports 47.50% under-attack utility with a 2.14% attack success rate, versus 35.36% utility and 51.07% ASR with no defense. Prompt-based (Spotlighting, Sandwich), filter-based (PromptGuard-2, PIGuard), and system-level (CaMeL, Progent, DRIFT) baselines either miss attacks or collapse task utility; on AgentDojo, ActGuard averages 59.85% under-attack utility and 0.42% ASR, with DRIFT the closest competitor (57.85% / 1.90%).
Is ActGuard's code available?
The abstract states that code is publicly available at github.com/binzhwang/ActGuard. The paper also releases the full algorithm, all prompts (action generation, verifier, attack payload), and default hyperparameters such as chunk size (80-180 chars), top-5 retrieval, and threshold tau=0.
What are ActGuard's main limitations and costs?
Overhead is roughly double the no-defense latency on AgentDyn (105.74 vs 53.50 seconds per task, Table 3) and about 26% more tokens. Recall is below 100% because payloads can span chunk boundaries; the tool prior is itself predicted from possibly contaminated context; the paper reports no repeated-run variance, no quantitative comparison with the closest concurrent methods (AttriGuard, CausalArmor, AgentSentry, ICON), and its audit prompt embeds security rules whose separate contribution to utility is not isolated.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2-5 (main AgentDyn comparison, AgentDojo comparison, localization analyses, adaptive-attack curves) are charts whose values cannot be read from extracted text; numeric verification relies on Appendix Tables 6-10, which do reproduce the key results.
- Listing 2/3 prompt text and parts of Appendix C appear interleaved or lightly garbled in extraction; a few sentences may be misassembled.
- The claimed public repository (github.com/binzhwang/ActGuard) was not inspected; the reproducibility rating assumes it exists and matches the paper.
- The paper cites several 2026-dated works (e.g., AgentDyn, DeepSeek-V4, GPT-5-mini, Gemini-3.1-Flash-Lite); their details were not verified offline.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.