When Tool Calls Succeed but Workflows Fail: Anomalies at the Agent-Tool Boundary
cs.AI · cs.DB · cs.DC · cs.SE
Quick verdict
Defines 8 agent-tool effect anomalies; 98k-tool MCP census shows no annotation can express the needed transactional contracts. Strong taxonomy, proofs pending.
Summary
The paper targets a correctness gap in LLM-agent workflows: individual tool calls can succeed while the external effects that survive contradict the workflow's intended resolution. The authors contribute an effect-history model that separates world events from the runtime's observations (Table 1); a catalog of eight anomalies — duplicated, missing, orphaned compensation, residue, premature externalization, contaminated speculation, conflicting externalization, phantom compensation — grouped into three families by construction, each paired with the boundary capability needed for general exclusion (Table 2); four guarantee profiles (unknown-safe, compensation-safe, speculation-safe, externally-mediated) with a reading of six recent runtimes as partial coverage (Table 3); and four guarantee boundaries — outcome resolution, conflict mediation, visibility control, atomic multi-tool release — argued to be unattainable through black-box invocation alone (Section 4.2). A census of 98,291 tools from the official MCP registry (snapshot 2026-07-27) shows the four advisory annotation hints are widely emitted (74.0% of tools serialize at least one) yet coarse: one signature covers 39.9% of tools, only 3.1% assert an applicable destructive operation, and none of the required capabilities is expressible (Table 4). Main limitations: catalog completeness remains a conjecture, the guarantee boundaries lack formal proofs, the census covers only the roughly half of remote targets that responded, and no field data quantifies how often the anomalies actually occur.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Eight recurring external-effect anomalies form three families whose membership is fixed by construction: the three actions possible under an unresolved outcome, the three ways an effect relates to its resolution point, and the two kinds of external participant.
- Excluding each anomaly in general requires a specific boundary capability, and four capabilities — authoritative outcome resolution, shared mediation, visibility control, and atomic multi-effect release — cannot be established through black-box tool invocation alone.
- In the official MCP registry (98,291 tools, snapshot 2026-07-27), the four advisory annotation hints are widely emitted but none of the required transactional capabilities is fully expressible; only A1 is partially addressable via idempotentHint.
- Current agent runtimes cover only fragments of the catalog under runtime-local assumptions, so the missing layer is reusable transactional contracts at the tool interface rather than better recovery mechanisms.
Strengths
Structurally derived, honestly scoped anomaly catalog
Section 3 derives the eight anomalies from three enumerable sources — the three safety-relevant actions under an unresolved outcome (re-issue/commit/compensate → A1–A3), the three effect-to-resolution relations (survival/timing/dependency → A4–A6), and two external participants (A7–A8). Section 3.3 explicitly excludes planning errors, read-side anomalies, security, misclassification, and liveness, and concedes completeness is a conjecture — unusual candor.
Events-vs-observations separation captures the real TP departure
The Table 1 vocabulary splits externalize from observe, and the A3 discussion makes the key point crisply: in classical transaction processing an in-doubt outcome is temporary because participants run recovery protocols, while a third-party tool runs none — unknown stays unknown until the tool offers a way out. This is the paper's sharpest conceptual contribution.
Guarantee boundaries anchored in classical impossibility results
Census scale and methodological transparency
Useful new vocabulary
Weaknesses
Central completeness claim is a conjecture with construction rhetoric
No frequency evidence for any anomaly
Census describes a biased reachable subset
Annotation presence is not shown to be deliberate declaration
Table 3 competitor characterizations rest on stated claims
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Review: When Tool Calls Succeed but Workflows Fail (arXiv:2609.15397)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What are the eight external-effect anomalies identified in this paper?
The paper catalogs eight ways agent workflows can leave external state inconsistent with the workflow's outcome: duplicated effects (A1), missing committed effects (A2), and orphaned compensations (A3), all caused by uncertain call outcomes; uncompensated residue (A4), premature externalization (A5), and contaminated speculation (A6), caused by workflow structure; and conflicting externalization (A7) and phantom compensation (A8), caused by interaction with concurrent executions or outside observers.
What did the MCP tool annotation census find?
Examining 98,291 tools from the official Model Context Protocol registry (snapshot 2026-07-27), the authors found the four advisory hint fields (readOnlyHint, destructiveHint, idempotentHint, openWorldHint) are widely present — 74.0% of tools serialize at least one — but coarse: one signature covers 39.9% of tools, only 3.1% assert an applicable destructive operation, and none of the transactional capabilities needed to prevent workflow anomalies is expressible through them.
Does this paper mean MCP tools are unsafe for AI agents?
Not exactly. The paper shows MCP's four boolean annotations are advisory and too coarse to support transactional guarantees (safe retries, compensation, staging, concurrency control). Tools may implement stronger semantics internally, but a runtime invoking them through the shared interface cannot rely on what the interface does not declare — that is an expressibility gap, not a claim that every tool misbehaves.
What do the authors propose instead of current MCP annotations?
Reusable transactional contracts at the tool boundary: idempotent re-issue returning the original outcome, status endpoints for authoritative outcome resolution, compensation declarations, staging with expiring holds, stable effect and resource identity with commit gating, resource-level commutativity declarations or mediators that serialize access, and visibility control before effects become externally observable.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Review based on the extracted full text; all tables were rendered inline and are readable, and no separate figures exist to assess.
- The released census artifact (github.com/flame-stream/mcp-annotation-census) was not inspected; its completeness is taken on trust.
- The 2026 arXiv references [10]-[23] describing competing runtimes could not be verified against their sources from this text alone.
- Census numbers are point-in-time (snapshot 2026-07-27) and cannot be independently re-verified here.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.