Compact-Memory LLM Agents via Online Max-Member Clustering and Atom-Aware Packing
Jiahe Geng, Jinpeng Wang, Kun Yuan
cs.AI
Quick verdict
Claims strong quality-token Pareto for compact memory (83% quality, 32% tokens on AMA-Bench); rigorous ablations, but margins vs nearest baselines are small.
Summary
This paper proposes RSM-full, an online clustered-memory pipeline for LLM agents operating under tight prompt budgets (roughly 2k–5k tokens). Incoming chunks are merged into 'atoms' via a cosine-gated max-member rule, and retrieved chunks are packed into the prompt grouped by atom rather than flat. On AMA-Bench (208 episodes), RSM-full reaches 83% of Full-Context quality at 32% of the token cost at a 4k budget and beats Online K-Means by +3.5–6.0 points under four-seed averaging; three-seed ablations attribute the gain mainly to the merge rule (+5.7 pp) and the grouped packer (+5.02±1.00 pp). On RealMem it improves on Budget-RAG (+0.69 pp) and A-MEM (+1.65 pp) while matching BM25-RAG. Limitations: the margin over the nearest baseline (Streaming-Proto) is small and non-significant single-seed; headline scores come from a non-official, aggregator-mediated GPT-5.4 judge; end-to-end results use one embedding stack; and some reported deltas do not reconcile with table means.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 5
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- RSM-full reaches 83% of Full-Context quality at 32% of token cost on AMA-Bench at a 4k budget, and under four-seed averaging beats Online K-Means by +3.5–6.0 pp across the ~2.6k–5k budget regime.
- Matched three-seed ablations attribute most of the gain to the cosine-gated max-member merge rule (+5.7 pp over Online K-Means and matched-τ DP-means, Table 3) and the atom-aware grouped packer (+5.02±1.00 pp over flat concatenation, Table 4).
- The compact-memory pattern replicates on RealMem: RSM-full beats Budget-RAG (+0.69 pp, p=.006), A-MEM (+1.65 pp, p<.001), and Streaming-Proto (+2.97 pp), while matching BM25-RAG (+0.27 pp, p=.47).
- Claims are explicitly scoped: raw LoCoMo and LongMemEval are boundary regimes where the advantage disappears, and the v1 retrieval rule is only benchmark-conditional supporting evidence.
Strengths
Matched causal decomposition is unusually rigorous
The 2×3 factorial (Table 3) with a matched-τ DP-means control (λ=√0.3, Kmax=16) cleanly separates the merge rule (+5.74 pp over K-Means, +5.45 pp over DP-means, both p<.0001) from generic growing-K behavior and from the retrieval rule (A−D null, p=.40); Table 5 shows K-Means with the same packer still trails by +4.90 pp.
Honest claim scoping, including negative results
The retrieval rule is demoted to conditional evidence (null on AMA, activates on 2/4 LoCoMo-Plus relation families), boundary regimes (raw LoCoMo, LongMemEval) are reported where the advantage disappears, BM25 equivalence is explicitly not claimed, and the negative Algorithm E result is published (Table 11, −0.76 pp NS).
Multi-seed robustness and diagnostics
Candid disclosure of failures and artifacts
Weaknesses
Headline margins over the nearest baseline are small
Low absolute quality levels
Primary scores depend on a nonstandard judge endpoint
Reporting inconsistencies across text and tables
Single embedding stack for all positive claims
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is RSM-full and how does it work?
RSM-full is an online clustered-memory pipeline for LLM agents under tight prompt budgets (roughly 2k–5k tokens). It merges incoming chunks into clusters called 'atoms' via a cosine-gated max-member rule, and assembles prompts by packing retrieved chunks grouped by atom (with atom headers and temporal ordering) rather than as a flat ranked list.
How much better is RSM-full than baselines?
On AMA-Bench (208 episodes), it reaches 83% of Full-Context quality at 32% of the token cost at a 4k budget, and beats Online K-Means by +3.5–6.0 percentage points under four-seed averaging across the ~2.6k–5k token regime. Ablations attribute most of the gain to the merge rule (+5.7 pp) and the atom-aware packer (+5.02±1.00 pp).
Does RSM-full work on all memory benchmarks?
No. The authors explicitly designate raw LoCoMo and LongMemEval as boundary regimes where the compact-memory advantage weakens or disappears; keyword-heavy settings can favor lexical (BM25) or hybrid retrieval. Positive end-to-end claims are scoped to AMA-Bench, RealMem, and grouped LoCoMo-Plus slices.
Is code or data available for this paper?
The paper states all experiments are reproducible from code and result files at the project repository and lists detailed artifact filenames (Appendix E). However, headline scores rely on a GPT-5.4-family judge accessed via an aggregator-mediated endpoint, which third parties may not be able to reproduce exactly, and no explicit public repository URL appears in the provided text.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure-embedded text (axis labels, legends, and annotations of Figures 1–4) is garbled by PDF extraction; body text and all tables are fully legible.
- The metadata abstract is cut mid-sentence, but the full abstract and the complete body through Appendix H and references are present; no truncation detected in the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.