The Irreversibility Budget: Fleet-Level Risk Accounting and Admission Control for Agent Operating Systems
Bardia Mohammadi, Laurent Bindschaedler
cs.AI · cs.DC · cs.OS
Quick verdict
A simulated risk ledger holds 1000-agent fleets under a shared cap where per-effect gates overdraw 48×; sound accounting, but pricing remains an open problem.
Summary
The paper argues that per-effect controls on LLM agents fail under composition: fifty individually capped procurement agents reacting to one shared price trigger can jointly overdraw the principal's risk tolerance while every local gate stays correct. It proposes the irreversibility budget — a cumulative per-principal account of residual value-at-risk maintained by a trusted runtime across an agent→workflow→tenant hierarchy. Effects are charged the γ-quantile of their residual loss after compensation and recovery; pricing sits below the agent's trust boundary, types are assigned by a canonicalization registry, and admission uses atomic reserve-confirm-cancel across all ledgers on the principal path. In a discrete-event simulation of the 50-agent fleet (300 seeded runs, 95% CIs), local gates overdraw in all runs at 2.4× tolerance on average and reach 48× at 1,000 agents, while the budget overdraws in no run, settles near 0.48×, and admits only $459k of a $1.5M fragmentation attack; typed charging executes 1.8× the value of a face-value cap. Trace analysis of 38,452 τ-bench and AgentDojo trajectories supports the shared-trigger premise: one planted injection propagates the same external effect across 84% of a heterogeneous fleet. The authors are candid that realized-loss safety depends entirely on conservative, dependency-aware pricing, which is unimplemented — misdeclared types reopen 77% overdraws and a five-tick detection lag still leaves 39%. This is a feasibility study, not a deployed system.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Per-effect authorization composes unsafely: local gates admit fleet-level overdraws averaging 2.4× a tenant's risk limit at 50 agents and up to 48× at 1,000 agents (§5.1–5.2).
- A trusted runtime can bound fleet exposure before commit by charging effects their residual-loss quantile against hierarchical per-principal ledgers with atomic reserve-confirm-cancel (§3–4).
- Typing effects by residual loss buys real utility: 1.8× the executed value of a face-value cap and 1.7× the strongest pooled cap when charges are honest (§5.4).
- Real agent traces exhibit shared-trigger correlation: one planted prompt injection propagates the same external effect across 84% of a heterogeneous fleet on average (§5.5).
- Realized-loss safety depends on conservative, dependency-aware pricing, which remains unsolved — misdeclaration and correlated recovery defeat additive charging (§5.4, §6).
Strengths
Cleanly isolated composition failure with structural explanation
RQ1 (§5.1) differs between arms only by the aggregate admission check, showing local gates overdraw in all 300 runs (2.4× tolerance on average) purely because no per-call gate can see the running sum; §5.2 shows the gap grows 2.4×→9.7×→48× at 50/200/1,000 agents.
Adversarial self-testing rather than win-only reporting
§5.2–5.4 stress the mechanism with a fragmentation attack ($1.5M split into sub-cap orders still admits only $459k), misdeclared refundable types (reopens $1.0M, 77% overdraw), and correlated recovery depression (0.75→0.25 causes 59% overdraw while the ledger believes itself safe).
Shared-trigger premise grounded in public traces, not just simulation
Unusually honest guarantee contract and limitations section
Reproducibility commitments
Weaknesses
Headline safety claim is near-tautological
All core results come from one synthetic generative model
The hard components are design sketches, and the paper's own results undermine one of them
Distributed budget authority deferred; microbenchmark has limited external validity
RQ5 fleet interpretation risks circularity
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This Agentic OS workshop paper (arXiv:2609.00275) proposes treating the irreversible external effects of LLM agent fleets as a first-class OS resource. The claim structure is careful: per-effect authorization composes unsafely — fifty individually capped procurement agents sharing a price trigger overdraw the tenant's $250k tolerance 2.4× on average, rising to 48× at 1,000 agents (§5.1–5.2); a trusted runtime maintaining a cumulative residual value-at-risk account per principal (agent→workflow→tenant) can deny the marginal effect before commit (§3–4); and safety in realized-loss terms depends on conservative, dependency-aware pricing that the authors explicitly leave open. Three contributions are claimed: the resource model (§3), the runtime mechanisms (§4), and the controlled feasibility study (§5).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the 'irreversibility budget'?
A cumulative per-principal account of residual value-at-risk that a trusted runtime maintains across an agent→workflow→tenant hierarchy. Every externally visible agent effect (payment, deletion, deploy, disclosure) is charged a quantile of its residual loss after compensation and recovery, and the runtime denies any effect whose charge would push the aggregate past the principal's authorized budget — analogous to how an OS accounts for memory.
What problem does the paper solve?
Individually authorized agents can jointly overdraw a principal's risk tolerance when a shared trigger — a price alert or a prompt injection — makes the same irreversible action attractive to many agents at once. In simulation, per-effect gates admit 2.4×–48× overdraws depending on fleet size, while a shared budget held exposure within the limit.
Is the system actually implemented or deployed?
No. The evaluation is a controlled discrete-event simulation plus an in-memory single-host ledger microbenchmark. Pricing, canonicalization, and shared-trigger detection are design proposals; the authors name conservative, dependency-aware pricing as the central open problem for a deployable design.
What evidence is there from real agents rather than simulation?
Trace analysis of 38,452 trajectories from the public τ-bench and AgentDojo benchmarks showing that external effects are task-determined rather than agent-idiosyncratic, and that one planted prompt injection propagates the same external effect across 84% of a heterogeneous fleet on average.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–4 appear in the extraction only as captions and stray axis-label text; the plotted shapes could not be visually inspected, though every number cited in this review appears in the prose.
- The stated GitHub release (simulator, seeds, trace analysis) could not be verified in this offline review; availability is reported as printed in §5.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.