What It Costs to Compose, Rebuild, and Correct Precomputed Memory
cs.CL · cs.LG
Quick verdict
Maps KV-cache/cartridge memory failure modes (composition, rebuild, correction) with rare pre-registration rigor; single-model, ≤24k-token scope limits reach.
Summary
This paper empirically maps when reusing precomputed model memory — a saved key-value cache from one prefill, or a trained compressed 'cartridge' — preserves correctness, on Llama-3.1-8B-Instruct. Three experiments: separately trained cartridges composed at serving time lose accuracy with count (26.25 raw points at eight memories; stripping duplicated BOS start-markers recovers 13.75, and 30% co-visible training protects nothing). Warm-started rebuild chains over 13 generations preserve quality only with replay of older training data: half replay matches full replay within the 10-point detectable margin at roughly half a fresh retrain's cost, while no replay is 15× cheaper and ends level with never updating. Serving corrections beside a stale memory fails: a lone relevant update rescues about a third of two-hop items, and no delivery policy (pasting, ordering, merging, tuned BM25 retrieval, or cache injection) comes within 64.3 points of the full-context reference at any pile size up to 512. The one large lever is wording: updates that name their question are used on over nine-tenths of items, but a false value that names the question is adopted at 0.877 — compliance, not recognition. Limitations: one model, lengths to 24k tokens, one seed for chains, mostly synthetic texts.
Review complete
Assessment at a glance
0
risk flags identified
● high 0
● medium 0
● low 0
5 strengths · 0 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Separately prepared memories recombine only at a growing penalty: composing 8 trained cartridges costs 26.25 points raw, and even after removing duplicated BOS start markers a net 12.5-point penalty survives 30% co-visible training.
- Warm-started rebuild chains over 13 generations preserve quality only with replay; half replay matches full replay within the detectable margin at roughly half a fresh retrain's cost, while no replay is 15× cheaper and ends level with never updating.
- No measured policy for serving corrections beside a static memory is workable: a lone relevant update rescues about a third of two-hop items and every policy ends at least 64.3 points below the full-context reference at all pile sizes 1–512.
- Update wording is the dominant lever: naming the question lifts use from 0.106–0.444 to over 0.92 regardless of delivery mechanism, and it works by compliance, since a false value that names the question is adopted at 0.877.
Strengths
Pre-registration with real teeth
Appendix A.4 documents an append-only record fixing analysis rules before data collection (registered tolerances, the two admissible chain shapes, the 10× compression bar set from record lengths before any training run); Amendment 94 committed the wrong-naming control's design, point predictions, and decision rule before any row existed, with provenance via content hashes over item manifests.
Controls that decompose, not merely detect
The three-way BOS control (Section 4.3) splits the 26.25-point 8-cartridge collapse into 13.75 mechanical + 12.50 residual; the initialization-matched retrain (Section 5.3) pins no-replay decay on accumulated history (−11.79, p = 0.0001); the token-matched padding control (Section 6.3) separates update content from bulk; the wrong-naming control identifies the mechanism as compliance rather than recognition.
Evaluator sensitivity measured, not assumed
Honest, bounded negative results with metered costs
Safety-relevant finding with a sharp mechanism control
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks what it costs to keep a **precomputed memory** — a saved key-value cache from one prefill pass, or a trained, compressed "cartridge" (self-study recipe of Eyuboglu et al., 2025) — correct as the underlying material changes. Three registered questions structure the work: can separately prepared parts recombine (Section 4), can warm rebuilds beat from-scratch cost (Section 5), and can corrections be served without rebuilding (Section 6), with the co-preparation premise measured in Appendix B. Every headline number comes from Llama-3.1-8B-Instruct (bf16, deterministic decoding); a Qwen3-8B check is explicitly descriptive. The claims are negative and bounded: preparation cannot be partial (two-hop visibility threshold τ = 0.82–0.92, a constant fraction across 1.5k–24k tokens, fitted slope −0.045, 95% CI −0.130 to +0.046), composed parts recombine only at a growing penalty, warm rebuilds preserve quality only with replay, and no measured correction policy is workable — every cell at least 64.3 points below the full-context reference on two-hop items at pile sizes 1–512.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 0 weaknesses, analyzed in detail
- 0 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is "precomputed memory" in this paper?
A model's saved reading of a body of material: either a key-value cache produced by a single prefill pass over the text, or a "cartridge" — a smaller cache trained via self-study to imitate the full cache. Either is reused across queries so the text need not be re-fed each time.
What is the paper's main practical advice for deployed systems?
Prepare each memory in one joint pass; never assemble from separately prepared pieces; take warm rebuilds only with replay (half replay preserved quality at roughly half a fresh retrain's cost); rebuild when the underlying facts change; if serving interim updates, keep them few (around 32 or fewer), rank or merge them, and word each to name the question it answers — but guard that channel, since false updates that name the question are followed nearly as readily.
Does removing duplicated BOS tokens fully solve cartridge composition?
No. Removing duplicated start-of-text markers recovers about half of the 8-cartridge collapse (13.75 of 26.25 points); a net penalty of 12.5 points remains, and 30% co-visible training did not reduce it in this setup.
Do the results generalize beyond Llama-3.1-8B?
The authors explicitly scope every verdict to Llama-3.1-8B-Instruct. A descriptive Qwen3-8B check repeats the naming direction at higher levels, but under a different item screen overlapping the primary one on only 119 items, so no model comparison is possible from the data presented.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures render in the extraction as raw axis-label strings; figure values were cross-checked against in-text numbers rather than visual inspection.
- No public code or data URL appears anywhere in the extracted text; codeAvailable:false reflects absence of a stated link, not a stated non-release.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.