Recoverability as a System Primitive for Long-Horizon AI Agents
cs.AI
Quick verdict
Proposes a testable contract governing how agents resume after failure; controlled demos are rigorous and honest, but small, self-built, vs weak baselines.
Summary
The paper proposes recoverability as a first-class system primitive for long-horizon AI agents: after interruption or failure, the system must explicitly decide which saved state (continuation source) and which recovery action (route) may be reused, or withhold automatic continuation. A formal contract (Section 3) defines candidates, evidence, supported states, and source-route permission sets, plus four testable requirements separating saved from eligible state, decision from execution, and both from restoration fidelity and task completion. An RPR reference architecture connects persistence, eligibility assessment, decision, enforcement, and audit across three layers and five stages. Three bounded runtimes support four controlled studies. In 64 source-selection runs, checkpoint-only selects an excluded newest checkpoint in 4/4 divergence challenges while both modes restore exactly; in 72 accounting runs, blind-retry overhead grows 2→4→8 while restoring modes incur 1, attributing retained prefixes to shared restoration; 20 paired file challenges replicate endpoint masking with model-generated files; and 18 matched authority cases with 84 bypass probes test event-time grants, withholds, and independently held policy evidence that exposes violations even after effects occur. Main limitations: validators and policies are supplied, scenarios are small and scripted with perfect scores on self-constructed contracts, source and route policies co-vary, costs are unmeasured, and implementations are frozen only in a private release.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A saved, exactly restorable state is not necessarily a valid continuation point; recovery requires an explicit, policy-checked source-and-route decision (continuation authority) that must be evaluated separately from restoration fidelity and final task success.
- Controlled challenges show that exact restoration plus task completion can conceal selection of a policy-excluded source (checkpoint-only: 0/4 contract-gold in Study A divergence cases and 0/20 in Study C, with 16/16 and 20/20 exact restoration and passing invariants).
- Retained validated progress is attributable to shared checkpoint restoration rather than the recoverability decision itself (Study B: restoring modes retain 1/3/7 validated actions at 1 extra attempt; blind-retry overhead grows 2/4/8).
- Permission must constrain the action at event time, and independently held policy evidence (a caller-retained pre-client configuration snapshot) can expose policy violations even after an effect has occurred (Study D, rounds 12 vs 13).
Strengths
Clear formal separation of recovery concerns
Section 3 cleanly splits latest stored (s_t) from latest eligible (l_t), decision from enactment, and both from restoration fidelity and endpoint completion; Table 1 makes decision-correctness orthogonal to task success, and the four requirements in §3.5 (including required withholding in both directions) are testable and implementation-agnostic.
Unusually honest evidence scoping
Table 10 maps each requirement to the evidence provided and its explicit boundary; §8 and §A.2 enumerate six qualifications of the Study D record (e.g., enforcement probes lack per-row manifests; exited-process OS codes are not re-observable); the threat model in §5.3 states precisely what is out of scope (spawn mechanism, OS injection, post-hoc byte tampering).
Well-constructed endpoint-masking counterexamples
Independent policy evidence experiment (Study D)
Weaknesses
Naive baselines; no external system comparison
Constructed circularity at tiny scale
Source-route co-variation confound
Artifacts not publicly available
Unverified model endpoint in Study C
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv:2609.13672, cs.AI) proposes **recoverability** as a first-class system primitive for long-horizon agents: after degradation, the system must explicitly select an evidence-supported continuation source and a permitted recovery route — or withhold automatic continuation — under a decision that governs execution and remains checkable afterward. Section 3 formalizes this: a candidate set B_t under a strict recency order, a Supported predicate binding evidence and policy, latest-stored (s_t) vs latest-eligible (l_t) queries, route sets partitioned into continuation vs withholding, and a decision function ρ whose output must belong to the policy-correct set D_t. Four requirements (§3.5) make the contract testable, including required withholding in the withhold-required direction so a system that always stops cannot look correct. Table 1 is the conceptual pivot: final-invariant success and decision correctness are orthogonal, so 'restored exactly + task finished' cannot certify a recovery. The claim is deliberately modest — within supplied policies and a declared trust model, this is a common testable interface, not discovered trust.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What problem does this paper solve?
AI agents save checkpoints but tend to resume blindly from the newest saved state, which may have failed validation or lost its approval. The paper formalizes 'recoverability' as a system-level decision: pick an evidence-supported checkpoint plus a permitted next action, or refuse automatic continuation — and score that decision separately from 'were the bytes restored exactly' and 'did the task finish'.
What are the main experiments and results?
Four controlled studies: 64 source-selection runs where a restore-latest baseline picks a policy-excluded checkpoint in 4/4 divergence cases while both restoring modes restore state exactly; 72 accounting runs showing checkpoint restoration (not the recovery decision) retains validated work — blind retry repeats 2/4/8 steps versus 1 extra attempt for restoring modes; 20 paired real-file challenges reproducing the same endpoint-masking with model-written documents (0/20 vs 20/20 correct source); and 18 matched permission cases plus 84 bypass probes testing event-time grants, withholds, and tamper-evident independent policy evidence.
Is the code or data available?
No. The paper freezes deterministic artifacts (git tags, commits, a SHA-256 archive hash, run manifests) but states they are retained in a private versioned release candidate and 'should not be read as public availability with this manuscript'.
What are the biggest limitations?
Validators and policies are supplied rather than learned; scenarios are small and scripted, so 100% scores only show conformance to the authors' own contracts; source and route policies change together in the checkpoint studies so routing is not isolated; there are no cost measurements; and the trust model assumes an honest spawn mechanism and excludes OS-level compromise — violations are detected after the fact, not prevented or rolled back.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt (80k chars) with five '[...sampling gap...]' markers; parts of §3's opening, §4.3–4.4, the Study A design opening/Table 5, mid-§6.5, and references [5]–[11] were skipped by sampling and were assessed only via surrounding text.
- Figures 1–5 appear in the extraction as flattened diagram text (concatenated labels); visual layout and rendering quality were not assessable.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.