Self-Cleaning and Captured Anyway: One Measured Primitive for Error in a Store an Agent Writes to Itself, and What a Falling Score Actually Measures
Wenhui Chen, Jianlin Chen, Ziyao Lin, Chi Man Vong
cs.CL · cs.AI
Quick verdict
Agent self-written stores end bimodally, predicted by a parameter-free copy function; rigorously preregistered but narrowly scoped, artifacts still private.
Summary
This paper studies what happens when a language-model agent writes its own answers back into the retrieval store it later reads from. Because the store is append-only, reachable states have a hard ceiling at (n−1)/n = 0.9938, and outcomes at f0=0.9 are bimodal: 173 of 220 runs end captured, 39 self-clean, and the interval between holds only 3.6% of runs against a uniform 20.6%, so the pooled mean describes just 8.2% of runs (median 68.2%). A single measured, parameter-free copy function γ(φ) predicts each run's direction: its sign relative to 1/k is correct on 353 of 360 Wikidata-fact runs and 39 of 40 synthetic runs. Four pre-registered interventions show timing of grounding dominates its fraction (0.20/0.33/0.40 vs 0.47), a consistency gate drives all models to 0.993 capture, and provenance marks do nothing unless correlated with truth. An embedding retriever displaces capture by 0.500, scoping results to uniform retrieval. At frontier scale, pooled capture is 0.850, with claude-sonnet-4.5 captured on 20/20 seeds against the authors' registered <0.5 prediction. Limitations: a small synthetic apparatus with small open-weight models, no mechanism for the bimodality or the step-2 decision, and the artifact repository remains private until acceptance.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Outcomes of the self-write loop are bimodal — 173/220 captured, 39/220 self-cleaning, 3.6% between modes vs 20.6% uniform — so the pooled mean describes only 8.2% of runs while the median describes 68.2%.
- One measured, parameter-free copy function γ(φ) suffices: the sign of γ̂−γcrit (γcrit=1/k at r=0, w=1) predicts each run's edge on 353/360 Wikidata runs and 39/40 synthetic runs.
- A falling exact-match score under contamination measures lost determinism, not lost information (pass@8−pass@1 = +0.383 on stores nothing was removed from), i.e. score = C·s.
- At matched grounding budget, timing dominates fraction (0.20/0.33/0.40 vs 0.47 ungrounded, monotone in 5/5 models); a consistency admission gate drives every model to 0.993 capture by helping the false majority; provenance marking moves nothing unless the mark correlates with truth.
- The seed is the correct resampling unit (design effect 3.75 pooled, 1.14–1.47 paired), all thresholds are scoped to uniform retrieval (embedding ranking displaces capture by 0.500), and at frontier scale pooled capture is 0.850 with claude-sonnet-4.5 captured on 20/20 seeds.
Strengths
Pre-registration and grading discipline at unusual depth
19 preregistrations with commit-frozen criteria; 87 graded rows in Appendix W, 37 carrying adverse statuses (withdrawn/failed/self-correcting/undecidable); failures reported as failures (P-S3 magnitude bar, P-B1/P-B3 budget criteria, arm A transport), including criteria that failed in the authors' favour.
Parameter-free predictive primitive validated on real facts
sign(γ̂−γcrit), γcrit=1/k at r=0, w=1, predicts drift direction on 353/360 Wikidata runs vs 39/40 synthetic; the out-of-sample holdout (Table 19) cuts error to 1.5–1.8 seed units vs arm A's 12.1, addressing the transport critique in Bareinboim–Pearl terms.
Objection-killing controls run at zero or trivial API cost
Self-refutation ledger kept as part of the argument
Transferable methodological result
Weaknesses
Narrow external validity, acknowledged but structural
Thin headline cells
No mechanism for either central phenomenon
Artifacts unverifiable at publication time
Derived-but-untested structure
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks what happens when an LLM agent writes its answers back into the retrieval store it later reads from — a closed loop at deployment time — under two pinned structural facts: the store is append-only and tenure is effectively unbounded. From this the authors derive a hard ceiling f ≤ (n−1)/n = 0.9938 at their apparatus (n0 = 10, T = 150) and show the outcome is a choice between two edges, not a decay. The registered claims are: (1) outcomes are bimodal and the interval between modes nearly empty; (2) one measured, parameter-free copy function γ(φ) predicts which edge each run reaches; (3) grounding timing dominates grounding fraction at matched budget; (4) falling exact-match scores measure lost determinism, not lost information (pass@8 − pass@1 = +0.383); (5) results are scoped to uniform retrieval after their own embedding-ranker falsifier fires; plus a transferable methodological result on seed-level resampling. The paper explicitly disclaims priority, transfer to training-time model collapse, and any incidence claim about deployed stores.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'Self-Cleaning and Captured Anyway' mean?
When an LLM agent answers from a store it has itself written to, each run ends in one of two extremes: 'self-cleaning' (the store drifts back toward truth) or 'captured' (the agent adopts the store's false majority, ending near the append-only ceiling f=(n−1)/n≈0.9938). Almost no runs land in between — 8 of 220 at f0=0.9 — so a falling average score does not describe what individual runs actually do.
What is the copy function γ(φ) in this paper?
It is the measured probability that the agent copies a false value when a fraction φ of the retrieved records carries that falsehood. With no fitted parameters, whether γ is above or below the threshold 1/k predicted which extreme a run reaches on 353 of 360 runs over real Wikidata facts (and 39 of 40 synthetic runs).
Is this the same as model collapse?
No. Model collapse is about training on generated data across generations. This paper studies a deployment-time loop inside one agent's memory store, which is append-only — nothing is ever deleted — so no steady-state occupancy exists and the mathematics differs from the retraining setting. The authors explicitly do not claim transfer to generational retraining.
Were the experiments run on real LLMs or only simulations?
Both. Five small open-weight models and one frontier model (claude-sonnet-4.5) ran the actual loop over roughly 7,244 total runs, including an arm on 36 real Wikidata facts. Scripted simulations — replaying the loop with the measured γ curve, or with a state-independent coin — served as controls and required no API calls.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Sampled excerpt: middle sections (parts of §4.5–§4.8, Appendix L detail, and several appendices) were skipped by the sampler and marked '[...sampling gap...]'; this review cites only content present in the provided text.
- The artifact repository is stated by the authors (§5.2) to be private until acceptance, so code/data availability could not be independently verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.