AgentAudit: An Open, Extensible Framework for Full-Lifecycle Trust Evaluation of AI Agents
Shrey Nag, Sachita, Abhishek Kumar Singh, Lipi Goel, Rajeshwar Singh Janwar
cs.AI
Quick verdict
Claims full-lifecycle trust auditing of LLM agents (10 dims, failure attribution); promising design but single-run results judged by the top-scoring model.
Summary
AgentAudit is an open-source framework that audits the complete execution trace of tool-using LLM agents rather than only final task success. It attaches post hoc to a recorded trace and scores ten dimensions — instruction integrity, planner, memory, tool selection, tool invocation, tool correctness, alignment, tool faithfulness, security (six attack categories) and execution integrity — plus behavioural classification (Safe_Correct, Unsafe_Compliance, Safe_by_incompetence, Partial_success) and failure attribution. Scores aggregate into a weighted Composite Trust Score (CTS) with a critical-failure cap at 30 when security, tool faithfulness or execution integrity fall to 29.9 or below. In the validation run, five models (GPT-5, Claude Sonnet 5, Sarvam 105B, Llama 3.3 70B, Gemini 2.5 Flash) each completed nine tasks across six simulated environments: Claude Sonnet 5 averaged 95.1, GPT-5 80.6, Sarvam 57.6, Llama 45.7, Gemini 22.6. The authors argue the classification distinguishes malicious compliance from capability failure, a difference pass/fail benchmarks hide. Key limitations: a single fixed judge (Claude Sonnet 5) scored all traces while also being the top-scored model; one trial per model-task pair with no variance estimates or statistical tests; module and attack weights are acknowledged as uncalibrated defaults; and behavioural labels on benign tasks (Unsafe_Compliance on Trip Planner) sit awkwardly with their definitions. Code is available on GitHub.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Existing benchmarks (AgentBench, AgentDojo, ASB, MCP-SafetyBench, TruthfulQA) each evaluate only a single facet, whereas AgentAudit is a unified execution-trace audit covering planning through security with per-stage failure attribution (Sections VI, IV.E.3).
- Behavioural classification separates Unsafe_Compliance from Safe_by_incompetence, exposing trustworthiness differences that task pass/fail scores hide (Sections IV.E.2, VII.D).
- On nine tasks, Claude Sonnet 5 (mean CTS 95.1) and GPT-5 (80.6) lead, with Sarvam 105B (57.6), Llama 3.3 70B (45.7) and Gemini 2.5 Flash (22.6) failing for different reasons: malicious compliance, incapability, and tool-use failure respectively (Tables I–III).
- Because AgentAudit only reads recorded traces, it can evaluate any LLM-based agent without constraining internal implementation (Sections III.A, VIII.C).
Strengths
Trace-level failure attribution fills a real gap
Section IV.E.3 and Table III surface distinctions pass/fail benchmarks conflate: Sarvam 105B is Unsafe_Compliance on 4/9 tasks (malicious compliance) while Llama 3.3 70B is Safe_by_incompetence on 5/9 (capability gap) at broadly similar composite scores (57.6 vs 45.7) — Section VII.D argues this correctly.
Candid methodological self-reporting
Section VII.E discloses the single-trial design, that the judge is one of the evaluated models, the Gemini 2.5 Flash-for-Pro quota substitution, and pre-run harness fixes (judge budget 4096→8192 tokens, missing Retry-After backoff, a cascade-skip bug) — unusually transparent for a benchmark paper.
Critical-failure cap prevents weighted-average masking
Decoupled, framework-agnostic architecture with open code
Weaknesses
Judge circularity
No statistical treatment
Uncalibrated weighting
Behavioural taxonomy breaks on benign tasks
Validation far thinner than claimed scope
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
AgentAudit (arXiv 2609.09875), produced under a MeitY Digital India internship, is an open-source framework that audits the complete execution trace of tool-using LLM agents instead of only final task success, positioning itself against single-facet benchmarks: AgentBench (task completion), AgentDojo and ASB (attack success/failure), MCP-SafetyBench, TruthfulQA (Sec. VI). It scores ten post-hoc dimensions — instruction integrity, planner, memory, tool selection, tool invocation, tool correctness, alignment, tool faithfulness, security (six attack categories), and execution integrity (Sec. IV) — and adds behavioural classification (Safe_Correct / Unsafe_Compliance / Safe_by_incompetence / Partial_success) plus failure attribution (Sec. IV.E). A weighted Composite Trust Score aggregates results, with a critical-failure cap: security, tool faithfulness or execution integrity at 29.9 or below caps CTS at 30 (Sec. IV.F), which explains the exact-30 cells for GPT-5 on Email and Jailbreak in Table II.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is AgentAudit and how does it differ from AgentBench or AgentDojo?
AgentAudit is an open-source framework that audits the full execution trace of a tool-using LLM agent after the task ends, scoring ten stages — instruction integrity, planning, memory, tool selection, tool invocation, tool correctness, alignment, tool faithfulness, security and execution integrity — plus failure attribution. AgentBench measures task completion and AgentDojo/ASB measure whether attacks succeed; neither pinpoints which pipeline stage caused an outcome.
Which models scored best in the AgentAudit paper?
Across nine tasks, Claude Sonnet 5 averaged 95.1 and GPT-5 80.6 on the 0–100 Composite Trust Score; Sarvam 105B (57.6), Llama 3.3 70B (45.7) and Gemini 2.5 Flash (22.6) trailed. Important caveat: every trace was judged by Claude Sonnet 5, which is also the top-scored model, and each model-task pair ran only once, so self-preference bias and run variance cannot be excluded.
What does Safe_by_incompetence mean in AgentAudit?
One of four behavioural classes: the agent failed to understand the task (or the malicious instruction) and therefore avoided it. The framework stresses this is not genuine robustness, unlike Safe_Correct, where the agent recognizes and refuses the malicious request; the other classes are Unsafe_Compliance (executes the malicious instruction) and Partial_success.
Is the AgentAudit code available?
Yes. The paper states code is available at github.com/ShreyNag/AgentAudit and describes integrations with OpenAI Agents SDK, LangGraph, CrewAI, AutoGen and MCP-based agents; evaluation reads only a recorded execution trace, so any agent producing such a trace can in principle be audited.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The weighted-aggregation formulas for the Security Score (Sec. IV.D) and Composite Trust Score (Sec. IV.F) are rendered as equation images; only 'subject to,' was extracted — the exact functional form is inferred from the surrounding weight tables, though the cap rule (≤29.9 → capped at 30) is stated in prose.
- Figures 1–12 are referenced but their content is not in the extracted text; all assessments rest on the tables and prose.
- Table III's per-cell attribution tags wrap onto separate lines in the extraction; classifications were cross-checked against the Section VII.D prose counts (Sarvam 4×Unsafe_Compliance, Llama 5×Safe_by_incompetence, etc.) and are consistent, but individual cells carry minor misalignment risk.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.