Invalidation Contracts for Cross-Episode Agent Memory
cs.AI
Quick verdict
Version-stamped invalidation contracts keep agent memory drift-safe; solid 7-model study on synthetic APIs; model compliance, not the protocol, binds savings.
Summary
The paper introduces 'invalidation contracts': a protocol layer in which API servers attach cacheability hints, per-table version stamps, structured reload diffs, dependency vectors, a knowledge graph, and a rules fingerprint (Levels 0-6) to recovery suggestions, so LLM agents that cache fixes across episodes can evict exactly the stale entries when server-side data drifts. A savings equation decomposes realized savings into validity (protocol-controlled, model-independent) and compliance (model-controlled). Evaluation spans seven models, three serving paths, two synthetic domains (an Acme/Stripe billing API and a recipe API), roughly 9,432 episodes in 250 runs with three seeds per drifted cell. Row-level invalidation (A2D) raises compliance by 0-66.7 percentage points over naive memory (55.6-66.7 on three models), yields eviction precision 1.00 versus 0.25 for table-level eviction, and recovers 29-33% of baseline token cost on four of seven models; table-level invalidation drops post-drift first-try rates to 0% on five of seven. Claude Sonnet 5 exhibits 'input-schema conservatism', applying rewrite hints at 55-60% but refusing add-a-field hints (10%). Limitations: both domains and drift schedules are author-built, so generality is untested; scoring denominators are 9-10 episodes with seed standard deviation up to 18.1 points and no significance tests; the L2 calibration runs underpinning the fixed compliance channel are not archived; prompt sensitivity is unexamined.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Validity is a protocol property: version-stamp checks are deterministic and produced identical results across seven models and three serving paths with zero contract failures in roughly 9,432 episodes.
- Compliance is a model property: identical wire bytes yield 100% first-try compliance on Claude Haiku 4.5 but 11% or below on Claude Sonnet 5, which refuses add-a-field fixes due to input-schema conservatism.
- Row-level invalidation (A2D) raises compliance by 0-66.7 percentage points over naive memory and recovers 29-33% of baseline token cost on four of seven models, while table-level invalidation (A2) drops post-drift first-try rates to 0% on five of seven.
- Stamp-plus-vectors detection dominates both lazy vector-only and blanket-clear eager policies on the rule stream, alerting one episode earlier at about 0.2% token cost (Table 5).
- A cheap compliance preflight (tens of API calls per model and action type) should gate any cross-episode memory deployment, since no protocol level can cross a model's conservatism boundary.
Strengths
Actionable factorization of savings
Equation 1 (realized savings = validity x compliance) cleanly separates what an API provider can engineer (validity, protocol levels) from what the model vendor controls (compliance), turning 'memory does not pay' into a diagnosable two-factor problem (§3.6); the A1 vs A2 vs A2D ladder isolates the value of precise invalidation specifically.
Exemplary measurement hygiene section
§4.1 pre-registers the stale-injection slot exclusions by fixed episode index, dissects the starvation artifact where an exhausted gemini-3.5-flash run reads 1.00 eviction precision at A2, and keeps completed-episode denominators visible; §6.4 reports that the pre-registered falsification criterion on drift frequency actually fired.
Adversarial self-critique of the protocol itself
Reproducibility infrastructure
Negative results reported in the same detail as positives
Weaknesses
Both domains are synthetic and author-built
No statistical inference despite tiny scoring denominators
Foundational compliance-channel calibration is unarchived
Key ablations never isolated
Headline savings describe the compliant subset
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper proposes **invalidation contracts**: a protocol layer in which an API server attaches machine-readable metadata - a `cache_hint`, a dependent-table name, response-level `table_versions`, structured reload diffs, Level-4 dependency vectors, a Level-5 knowledge graph with pre-reload edge walks, and a Level-6 `rules_version` fingerprint - to every recovery suggestion, so an agent caching fixes across episodes evicts exactly the stale ones (§3). The analytic core is Equation 1: realized savings = validity x compliance(model, protocol, action). Headline claims: validity is deterministic and model-independent (zero contract failures in ~9,432 episodes); compliance is a model property - identical wire bytes give 100% first-try compliance on Claude Haiku 4.5 but at most 11% on Claude Sonnet 5 (input-schema conservatism); row-level invalidation (A2D) raises compliance 0-66.7 pp over naive memory (A1) and recovers 29-33% of baseline token cost on four of seven models, while table-level invalidation (A2) drops post-drift first-try rates to 0% on five of seven.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is an invalidation contract for agent memory?
A set of machine-readable fields an API server attaches to error-response recovery suggestions - a cacheability hint, per-table version stamps, row-level reload diffs, and at higher levels dependency vectors, a knowledge graph, and a rules fingerprint - so an LLM agent that caches fixes across episodes can tell exactly which cached entries are stale when server data changes, instead of discovering staleness by replaying errors.
How much does the invalidation contract save?
On four of the seven tested models it recovers 29-33% of the no-memory baseline token cost; naive caching without invalidation already recovers 10.3-28.2%. Three models benefit far less (22.5%, 8.7%) or lose about one point versus naive memory, because they ignore the cached fixes.
Why did Claude Sonnet 5 benefit least from the protocol?
It showed 'input-schema conservatism': it applied hints that rewrite existing values 55-60% of the time but refused hints requiring a new field in the request about 90% of the time (10% compliance), so the protocol's correct invalidation never translated into applied fixes.
Is the code and data for this paper available?
Yes. The server implementation is at github.com/arquicanedo/self-reflective-apis and the client evaluation harness at github.com/beining1008/cross-episode-memory-client, with per-episode run artifacts and the tabular extraction behind the paper's tables archived in the client repository.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 3-6 appear in the extraction only as axis-label and legend text; their plotted values were taken from the prose and tables, which cover every number cited in this review.
- The abstract as supplied in the metadata ends mid-word, but the identical sentence completes in the body text ('five of seven'), so no abstract content is missing.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.