MoM: Memory of Memory
cs.CL
Quick verdict
P-Mem commits values at write time while keeping provenance, cutting stale answers and surviving revision chains; gains lean on a self-built benchmark.
Summary
The paper argues that LLM-agent memory should commit each fact's current value at write time while retaining displaced values as typed provenance, rather than storing everything for query-time reconstruction (which resurfaces stale values) or overwriting CRUD-style (which makes errors irreversible). It instantiates this as Provenant Memory (P-Mem): a typed provenance graph with evidence/version/operation/tombstone nodes and six operations (ACCEPT, CONTEST, REJECT, REVOKE, RESOLVE, NOOP), plus soft key identity and deterministic materialization. On LongMemEval-s, a turn-level read with graph-guided pruning matches the strongest retrieval baseline (52.6% vs 50.8% HippoRAG-v2, within noise; 45.8% for MemoryOS) at roughly 4x fewer context tokens and cuts the knowledge-update stale-answer rate from 19.4% to 10.9%. On a self-built ALFWorld-Revision benchmark, write-time commit stays at 100% across revision chains where a query-time read falls to 25% at eight revisions; rollback recovers retracted errors 100% versus 0% without retention. On MemoryAgentBench FactConsolidation, iterative reads lift multi-hop accuracy to 53% versus 14% for an order-tagged flat store at the 6k scale. Main limitations: on natural histories the write-time materialized path still loses to read-time pruning; most non-ACCEPT operations appear only in templated probes; the 4x token saving is a retrieval-granularity effect rather than a property of the graph; and no code release is mentioned.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Agent memory should commit each fact's current value at write time while retaining displaced values as typed provenance, a combination no existing design family occupies (Table 1).
- Graph-guided turn pruning cuts the knowledge-update stale-answer rate from 19.4% to 10.9% on LongMemEval-s while matching the strongest retrieval baseline's accuracy at roughly 4x fewer read tokens.
- Write-time commit stays at 100% current-value accuracy on revision chains up to length 8 where a query-time read collapses to 25%.
- Retaining displaced values makes committed errors reversible (100% rollback recovery vs 0% without retention) and lets CONTEST/RESOLVE hold a genuine conflict open and settle it once (97.9% post-resolution accuracy).
- On natural (non-keyed) histories, write-time materialization still loses to read-time pruning, so extraction coverage and key canonicalization, not the state mechanism, are the current bottleneck.
Strengths
Formalizes an under-articulated design space
Table 1's two axes (when the current value is settled; what happens to the displaced value) cleanly position archival (RAG/long-context/MemGPT), CRUD (Mem0), and bitemporal-KG (Zep/Graphiti) families and expose the unoccupied commit-plus-retain corner, making the contribution falsifiable rather than rhetorical.
Exceptional statistical hygiene
McNemar 8-0 (p<0.001) for the paired prune effect; paired bootstrap CIs on the 500-question set (+6.8 [+2.4,+11.2], p=0.001 vs MemoryOS; +1.8 n.s. vs HippoRAG-v2, correctly called a tie); binomial ±10-point margins stated with explicit identification of which gaps exceed them (41 vs 14 exceeds; 11 vs 8 at 262k read as tie).
Controlled mechanism probes rather than end-to-end hand-waving
Honest negative results and self-undercutting attributions
Fair baseline treatment with scale checks
Weaknesses
Marquee results live on a self-built, templated benchmark
Operation vocabulary largely untested outside probes
Key fragmentation on natural text remains unsolved
No code or data release
Write-side economics asserted, not measured in deployment
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper addresses a narrow but real question: for a long-horizon LLM agent, how should memory answer "what currently holds" when facts change? Section 1 and Table 1 formalize two design axes - whether the current value is settled on arrival or at query time, and whether the displaced value is retained or erased - and position archival memory (RAG, long-context, MemGPT/MemoryOS), CRUD memory (Mem0), and bitemporal KGs (Zep/Graphiti) in three corners, claiming the commit-plus-retain corner is unoccupied. P-Mem instantiates it: an append-only typed provenance graph (Section 3.2) with six operations, a deterministic materialization exposing one head per key, soft key identity (0.72 cosine plus a one-word check, Section 3.3), and two read paths. The claims are explicitly validity-centric (staleness, reversibility, conflict) rather than accuracy-centric, and the paper is commendably upfront about this framing.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Memory of Memory (MoM) and P-Mem?
MoM is a design principle for LLM-agent memory: commit the current value of each fact at write time while keeping the values it displaces as typed provenance. P-Mem is its implementation as a provenance graph with six typed operations (ACCEPT, CONTEST, REJECT, REVOKE, RESOLVE, NOOP), so an agent reads one current value per fact instead of reconstructing the present from stored history at query time, while mistakes stay reversible.
How is P-Mem different from Mem0 or Zep/Graphiti?
Mem0-style CRUD memory overwrites old values, so a wrong update is permanent. Zep/Graphiti's bitemporal knowledge graph keeps superseded edges but arbitrates purely by timestamp and has no operation that reopens a committed value (the paper measures 0% recovery on retractions). P-Mem keeps one current value per key, records explicit supersession/conflict/retraction edges, and can roll back along the supersession chain.
What benchmarks were used and what were the headline numbers?
Three: LongMemEval-s (52.6% accuracy at ~3k context tokens vs 45.8% for MemoryOS retrieval at ~11k; knowledge-update stale-answer rate cut from 19.4% to 10.9% by graph-guided pruning), a new ALFWorld-Revision variant (write-time commit holds 100% on 8-deep revision chains where a query-time read falls to 25%; rollback recovers retracted errors 100% vs 0% without retention), and MemoryAgentBench FactConsolidation (iterative reads reach 53% multi-hop accuracy vs 14% for an order-tagged flat store at the 6k scale).
Is code available for this paper?
The paper text does not mention a public code or data release. It does give detailed implementation appendices: model versions (Qwen3-4B-Instruct-2507 agent, Qwen3-Embedding-4B retriever, Qwen3.6-35B judge), thresholds, seeds, sample sizes, benchmark construction templates, and cost accounting, which would allow a determined reimplementation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure diagrams (Figures 1-2) are not visually assessable from the text extraction; judgments about them rest on captions, tables, and body prose, all of which are present and consistent.
- The metadata abstract supplied with the task is cut mid-sentence ('displaced values are retai...'), but the full abstract appears complete at the start of the paper text, so no content was lost for this review.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.