CoMem: Collective-Individual Memory Synergy for Evolutionary Multi-Agent Systems
Chengxin Yu, Zhaoxin Fan, Faguo Wu, Hongwei Zheng, Yun Zhou, Zhiyu Li
cs.AI
Quick verdict
CoMem adds two-tier private/collective memory with promotion gates to multi-agent LLM systems, reporting gains on ALFWorld/PDDL; promising breadth, but internal number contradictions and no statistical testing weaken the evidence.
Summary
CoMem is a two-tier memory architecture for LLM-driven multi-agent systems, designed to counter 'memory pollution': noise accumulation in flat shared memories and behavioral homogenization from indiscriminate cross-agent writing. Each agent distills post-episode trajectories into concise private insights via an LLM distiller, pruned by a consecutive-miss rolling window. Entries are promoted to a shared collective pool only after sufficient use and above-baseline reward; pool entries carry an exponentially moving average utility that triggers purging. At decision time, agents retrieve in parallel from both streams, with online hierarchical clustering deduplicating collective results. On ALFWorld and PDDL, across AutoGen, DyLAN, MacNet, and CARD with a DeepSeek-V4-Flash backbone, CoMem improves success rates over the no-memory condition in all eight settings, averaging +8.02 points (ALFWorld) and +4.70 (PDDL); ablations show both tiers contribute, and Kmax/alpha sweeps show inverted-U sensitivity. Limitations: memory pollution is never measured directly; reported numbers contain internal contradictions (Table 2 column labels, k-sweep defaults conflicting with headline results); no variance or statistical tests accompany single-run success rates; hyperparameters are tuned per benchmark; no code, prompts, or Qwen3-32B validation numbers are provided.
Review complete
Assessment at a glance
5
risk flags identified
● high 2
● medium 2
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A two-tier memory architecture (private experience sedimentation, collective wisdom curation, parallel dual-stream retrieval) suppresses memory pollution in LLM-driven multi-agent systems.
- CoMem improves success rates over the No-memory condition in all eight framework-benchmark settings, averaging +8.02 points on ALFWorld and +4.70 on PDDL with DeepSeek-V4-Flash.
- Both memory tiers are structurally necessary: removing the collective layer costs 12.03 points and removing the private layer 9.34 points on AutoGen/PDDL.
- Conventional shared-memory baselines frequently underperform the No-memory condition due to noise accumulation and behavioral homogenization.
Strengths
Broad and honest baseline evaluation
Eight memory mechanisms (No-memory, Voyager, MemoryBank, Generative Agents, MetaGPT, ChatDev, G-Memory, CoMem) x four MAS frameworks (AutoGen, DyLAN, MacNet, CARD) x two benchmarks (ALFWorld, PDDL) in Table 1; notably, the paper candidly reports that conventional shared-memory baselines frequently underperform the No-memory condition (e.g., G-Memory 59.53 vs 67.78 under DyLAN/PDDL), a genuinely useful negative result.
Architecturally coherent anti-pollution design
The promotion gate (Nuse >= 3 and mean reward above moving baseline R-bar) plus EMA-based collective decay (alpha=0.5) plus miss-counter private pruning (Kmax=5/6) form a full lifecycle that plausibly addresses both noise accumulation and homogenization; component ablations (Table 2) show each tier matters.
Consistent positive deltas across all eight framework-domain settings
Real cost accounting attempt
Weaknesses
Contradictory and apparently mislabeled numbers across tables
No statistical rigor
Core 'memory pollution' claim never measured directly
Plug-and-play claim vs per-benchmark tuning
Reproducibility gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Preliminary Review: CoMem (arXiv:2609.15009)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is CoMem and what problem does it solve?
CoMem is a memory architecture for LLM-driven multi-agent systems. It combines per-agent private memory with a shared, curated collective pool: each agent distills its task experiences into short private insights and prunes unused ones, while only insights that have been used repeatedly and achieved above-average task scores are promoted into the shared pool, whose entries decay if their utility drops. Agents retrieve from both sources in parallel. It targets 'memory pollution' - shared memories filling with unverified trial-and-error noise and agents converging to indistinguishable, homogenized behavior.
How well does CoMem perform on ALFWorld and PDDL?
The paper reports consistent success-rate gains over the same framework without memory in all eight tested combinations of four multi-agent frameworks (AutoGen, DyLAN, MacNet, CARD) and two benchmarks (ALFWorld, PDDL), averaging +8.02 percentage points on ALFWorld and +4.70 on PDDL with a DeepSeek-V4-Flash backbone. However, the reported numbers contain internal contradictions between tables, and no variance or statistical tests are provided, so the margins should be read as preliminary.
What is 'memory pollution' in multi-agent LLM systems?
It describes how a blindly shared multi-agent memory pool degrades over time through two linked processes: noise accumulation (unverified trial-and-error records flooding the pool) and behavioral homogenization (agents overwriting each other's role-specific knowledge until behavior converges to a noisy average). The paper's evidence that this matters is that several flat shared-memory baselines, such as G-Memory and MetaGPT on some frameworks, score below the no-memory baseline.
Is code available for CoMem?
No code, prompts, or data release is mentioned in the paper. Hyperparameters are documented (retrieval depth k=3, miss window Kmax=5/6, EMA rate alpha=0.5, promotion threshold of 3 uses with above-baseline reward), and a second backbone (Qwen3-32B) is described qualitatively but without numbers. The method uses public benchmarks and open frameworks, but exact reproduction would currently require a full reimplementation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Text extraction is noisy around Figure 1, Figure 2, and Table 1 (embedded figure text interleaved with body text), so table values were cross-checked against prose where possible.
- Table 2's column-label swap is inferred from the paper's own cited drop magnitudes; if the extraction mangled headers, some flagged inconsistencies may be artifacts, though the k-sweep/Table 1 and Table 1/Table 2 conflicts involve consistent plain numbers in prose and appear genuine.
- Benchmark task counts were inferred from quantization steps of reported success rates; the paper never states them.
- Figures 3 and 4 are only partially recoverable from extracted axis text; exact token budgets and some data points could not be verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.