Fresh Memory, Stale Plans: Dependency-Scoped Validation for Distributed LLM-Agent Memory
Evan Chen, Shiqiang Wang, Christopher G. Brinton
cs.AI
Quick verdict
PlanFence ties plan validity to record lineage; evidence is protocol-clean and honest, but baselines fail by construction and scope is three workflow families.
Summary
The paper identifies 'stale-plan execution' in distributed LLM-agent teams: an executor can hold freshly propagated state yet act from a plan derived from superseded records. It proposes PlanFence, a protocol in which plans cite the exact public record IDs they used, tool wrappers declare the dependency set of each action, and the executor validates the plan's dependency frontier against owner-reported current versions immediately before an external call — replanning once on mismatch, then failing closed. In 30 live five-agent Qwen3.5 workflows with one injected revision, a freshness-only executor issues an obsolete action in 30/30 tasks, while PlanFence and centralized lineage each complete all 30 after one replan. In controlled replay (330 actions per cell, eight keys, AT&T trace, rho=4), every exact-lineage policy is safe, and PlanFence shows the lowest stall (230.8 ms) and traffic (8.1 KiB/action) among safe policies. PlanFence leads at high update rates (1.5-7.1x for rho>=4) while proactive metadata sync wins at low churn; against batched all-key validation it cuts stall by up to 67% and traffic by up to 90% at sparse dependencies, with the advantage vanishing when dependencies span the full keyspace. Limitations: three workflow families, 3-8 agents, a single model, historical network traces, and safety conditional on complete dependency declarations and benign owners, with no atomicity between validation and the external action.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- State freshness does not imply plan validity: an executor can read the latest shared record yet act from a plan derived from a superseded version (stale-plan execution), formalized as lineage validity in Eq. (1).
- Binding plans to exact public record IDs and validating only declared dependencies with owners before the action (one replan, then fail-closed) eliminates invalid actions: 0/330 in replay and 30/30 live tasks completed, matching centralized lineage safety.
- Coordination cost has a measured crossover: proactive metadata sync (K=1) has lower stall at low churn (rho<=1), while PlanFence leads at rho>=4 by 1.5-7.1x across loopback and three cellular traces.
- When actions depend on a sparse subset of shared state, dependency scoping beats equally-safe batched all-key validation on both stall (up to 67% lower at 128 loopback keys) and traffic (up to 90% lower), with the advantage vanishing at d=|X| (Table 6).
- The safety guarantee is conditional on exact parent links, complete tool-declared dependency sets, benign owners, and fail-closed handling, and is not atomic across owners or with the external action (Secs. 3.3, 6).
Strengths
Sharp problem formalization
Lineage validity (Eq. 1: F_a(x)=H(x) for all declared dependencies) cleanly separates executor state freshness from plan authorization; the invariant-vs-mechanism split (Secs. 1, 3.2-3.3) makes the contribution modular and explains why each condition is necessary.
Unusually complete and honestly-labeled baseline grid
Table 1 spans unsafe freshness (local replica, owner-head), unsafe delayed sync (K=2-16, explicitly 'a safety-cost sensitivity, not a safe headline competitor'), and five safe alternatives (centralized lineage, metadata sync K=1, per-key/batched all-key, majority/all-replica controls), all under identical schedules.
Statistical hygiene rare in this space
Consistent claim discipline
Mechanism-level ablations
Weaknesses
100% baseline failure is by construction
Safety rests on a strong trusted computing base
Validation is not atomic with the action
Narrow external validity
No artifact availability
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets a specific distributed-consistency hazard in LLM-agent teams: *stale-plan execution* — an executor holds current public state yet acts from a plan derived from superseded records. The claims are: (1) freshness does not establish plan authorization, formalized as lineage validity (Eq. 1: F_a(x)=H(x) for all declared dependencies); (2) PlanFence — plans cite exact public record IDs, tool wrappers declare dependency sets, owner-head validation at the action boundary, one replan, fail-closed — eliminates invalid actions at lower cost than safe alternatives in the measured regimes; (3) a crossover exists between proactive synchronization and action-time validation; (4) dependency scoping beats all-key checking for sparse dependencies. The abstract and Section 6 explicitly scope results as "controlled safety and systems-cost results, not general task-accuracy gains."
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is stale-plan execution?
A failure mode in distributed LLM-agent teams where an agent holds the latest shared facts (e.g., a revised requirement r4) but still executes an action whose arguments came from a plan derived from an older version (r3). The paper's point is that fresh memory state does not prove the plan authorizing an action is still current — the plan's derivation lineage must be checked.
How does PlanFence work?
Each plan records the exact public record IDs it was derived from; each tool wrapper declares which shared keys can affect the action. Before any external call, the executor asks each key's authoritative owner for the current version and checks the plan's dependency frontier against it. A mismatch triggers one replan and revalidation; any missing lineage, malformed owner response, or second change blocks the action (fail closed).
When is PlanFence better than synchronizing every update?
In the paper's controlled replay, proactive metadata synchronization has lower waiting time at low update rates (rho<=1), while PlanFence has 1.5-7.1x lower stall at high update rates (rho>=4). Versus checking all shared keys, PlanFence wins when actions touch a sparse subset — up to 67% lower stall and 90% lower traffic at 128 keys — and the advantage disappears when an action depends on the whole keyspace.
What are the main limitations of the study?
Evidence covers three workflow families, 3-8 agents, one model (Qwen3.5-35B), and pinned historical cellular traces; the 100% baseline failure rates come from schedules that deliberately stage the race. Safety assumes complete dependency declarations, exact parent links, and benign owners, and validation is not atomic with the external action. The authors explicitly claim no general task-accuracy gains.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2-6 are image-embedded; their data points are available only through captions and in-text annotations (e.g., Fig. 5's 67%/90% labels, Fig. 3's in-cell ratios), so values plotted but never stated in text (e.g., exact per-trace points in Fig. 2, curve values in Fig. 6) could not be independently verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.