Fortunate Recall: Ontology-Driven Memory Lifecycle Management for Persistent Coherence in LLMs
cs.AI
Quick verdict
Ontology-driven LLM memory lifecycle; pre-registered ablations credit generic metadata with correctness, ontology with calibration; LLM-judge-only validation.
Summary
The paper introduces Fortunate Recall (FR), a deterministic lifecycle policy layer for LLM agent memory. Each extracted fact is classified into a 10+1 behavioral ontology and receives category-specific decay, slot-key supersession, event-time validity, and retrieval routing as closed-form scoring over LLM-extracted metadata; a non-identifiability argument motivates the metadata basis of category, slot key, lifecycle state, and event-time anchor. The authors also contribute LifecycleBench, a 516-question synthetic benchmark (40 personas, 9 attack vectors) whose answers depend on lifecycle state. FR-Bank reaches 76.9% retrieval pass versus Mem0 (61%), A-MEM (65.3%), Memory-R1 (66.9%), and MemoryOS (70.5%), scores 75.2% on LongMemEval-S, and cuts end-to-end confabulation from Mem0's 45.1% to 22.4% over answered queries. A pre-registered ablation shows the generic lifecycle metadata—not the ontology—carries the correctness gain (Δ −1.7pp, not significant), while the typed layer halves confabulation (12.0% vs 24.2%, p<0.001); the decomposition transfers to the independent BEAM benchmark (46.8% vs 32.9%). Acknowledged limitations: all verdicts come from LLM judges without human validation, the benchmark is co-designed with the method, competitors run in shipped defaults, and FR abstains on 42.1% of queries.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The lifecycle metadata basis (category, slot key, lifecycle state, event-time anchor) is necessary and sufficient for lifecycle-aware scoring, and any scorer restricted to (semantic similarity, age) fails by construction (Theorems 1–3, 3', 3'').
- FR-Bank reaches 76.9% on LifecycleBench (vs Mem0 61%, A-MEM 65.3%, Memory-R1 66.9%, MemoryOS 70.5%) and 75.2% on full LongMemEval-S, so lifecycle policies impose no aggregate retrieval cost.
- A pre-registered ablation shows generic lifecycle metadata carries the correctness advantage (Δ −1.7pp, n.s.) while the behavioral ontology carries calibration, halving confabulation (12.0% vs 24.2%, p<0.001).
- The decomposition transfers to the independently built BEAM benchmark (46.8% vs Mem0's 32.9% over 280 questions), with the ontology's gain concentrated in contradiction resolution, though the confabulation half does not replicate there.
- The deterministic lifecycle layer costs 47μs median at k=20, and the ontology's granularity benefit saturates near seven policy clusters, so 10+1 is an interpretable point on a plateau, not uniquely necessary.
Strengths
Rare claim-narrowing honesty via pre-registered ablation
§6.5 shows the untyped lifecycle arm leaves correctness statistically unchanged (Δ −1.7pp, 95% CI [−6.0, +2.7], McNemar p=0.44), and the paper states this as its claim: generic metadata carries correctness, the ontology carries calibration (confabulation 12.0% vs 24.2%, p<0.001). The conclusion even says the claim is narrower than the submitted version.
External transfer on an independently built benchmark
§6.6 pre-registers the FR-vs-Mem0 comparison on BEAM (ICLR 2026): 131/280 (46.8%) vs 118 for the k=1 collapse and 92 (32.9%) for Mem0, with the ontology's correctness gain concentrated in contradiction resolution (29 vs 19 of 70, McNemar p≈0.006) — a benchmark the authors did not build.
Machine self-auditing of the judge and disclosed errors
Explicit core-vs-engineering separation with per-feature ablations
Dual-denominator confabulation reporting with latency profiling
Weaknesses
No human validation of any verdict
Benchmark–method co-design
Shipped-default competitors vs engineered stack
Retraction suppression is mostly downstream, not structural
High abstention as a deployment constraint
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a method paper with a bundled benchmark. Fortunate Recall (FR) attaches metadata to every extracted memory edge — behavioral category (10+1 ontology), slot key, lifecycle state (active/superseded/expired/retracted), event-time anchor — and applies deterministic policies (category-specific decay, slot-key supersession, event-time validity with anticipatory activation, category-aware routing) as a closed-form log-score (Eq. 2). §4 explicitly scopes the theoretical claims: representation-level impossibility for (σ, Δt)-only scorers (Thm 1), sufficiency/minimality of the basis (Thms 2–3 with four concrete witnesses in A.7.1–A.7.4), and convex-feasibility for calibration (Thm 4, instantiated on 99.8% of 1,566 cross-pairs). Empirical claims: FR-Bank's 76.9% LifecycleBench pass and 75.2% LongMemEval-S; an attribution split in which generic lifecycle metadata carries correctness while the ontology carries calibration; and transfer to the independently built BEAM benchmark. The conclusion states the narrow version outright ('That claim is narrower than the one we submitted').
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Fortunate Recall (FR)?
A policy layer for LLM agent memory that classifies each stored personal fact into one of 11 behavioral categories and applies category-specific lifecycle rules — decay rates, replacement (supersession), expiry by event date, and retrieval routing — as deterministic math over metadata extracted by an LLM, instead of treating all facts identically like a flat vector store.
How much does FR improve over Mem0?
On the paper's new LifecycleBench, FR-Bank reaches a 76.9% retrieval pass rate versus Mem0's 61%. End-to-end confabulation over answered queries drops from Mem0's 45.1% to 22.4%. On the independently built BEAM benchmark, FR scores 46.8% versus Mem0's 32.9% over 280 questions.
Does the behavioral ontology actually matter, per the paper's own tests?
Yes, but only for calibration: a pre-registered ablation found that removing the ontology while keeping generic lifecycle metadata leaves retrieval correctness statistically unchanged (−1.7pp). The ontology's measured benefit is halving confabulation (12.0% vs 24.2%, p<0.001) plus a localized correctness gain on contradiction resolution on BEAM.
Is the code and benchmark publicly available?
The paper states that the ontology, the 516-question LifecycleBench, code, pre-registration files, and raw per-question judgments are released as supplementary material and will be made publicly available upon publication; as of this preprint they cannot yet be independently downloaded or verified.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with marked [...sampling gap...] blocks; sections inside gaps (parts of the appendix proofs, full Appendix AD/AE text, and several F–Z appendices) were assessed only from content visible elsewhere in the excerpt, so appendix-level detail claims are correspondingly reduced in confidence.
- The metadata abstract is cut off mid-sentence ('while a'), but the embedded paper text contains the complete abstract, so no review judgment relied on the truncated metadata field.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.