MACE: Memory-Agent Co-Evolution with Adaptive Memory Graphs for Multi-Agent Systems
Kairui Yang, Kairui Yang, Minghao An, Xunkai Li, Xunkai Li, Ziheng Yi, Ziheng Yi, Zekai Chen, et al.
cs.LG
Quick verdict
MACE's memory-graph co-evolution tops 8 benchmarks (81.11% vs SAGE 78.97%), but single-run evals, sub-0.1pp margins and uncited baselines leave SOTA unproven.
Summary
The paper proposes MACE, a memory-agent co-evolution framework for LLM-based multi-agent systems. Its MemGoG structure packages collaboration traces into functional memory units (subgraphs of conditions, actions, and outputs) connected by typed relations such as support, conflict, and repair. A three-stage loop adapts both memory organization and use: a Memory Composer selects units under a token budget, an Agent-Memory Coupler renders role-specific instructions or checklists, and a Feedback Co-Evolver updates unit utilities and relations from execution outcomes. Four controlled studies motivate the design: functional units retain about 83% of internal dependencies versus 68% for matched nonfunctional groups, and outer relations raise cross-experience coverage from 52% to 69%. Across eight benchmarks totalling 4,728 queries (MMLU, MMLU-Pro, GSM8K, AQuA-RAT, HumanEval, LiveCodeBench v6, TabFact, TAT-QA), MACE reports the top score on all eight, averaging 81.11% versus 78.97% for the strongest baseline, SAGE. Key limitations: all results are single runs without variance or significance testing, several winning margins are below one example's worth of accuracy (GSM8K by 0.08 points), key baselines (SAGE, MAGMA, GraphSearch, R-GFM) are uncited and some rows are imported under different protocols, tables show minor internal inconsistencies, and the serving LLM backbone is not identified.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Grouping collaboration-trace dependencies into functional memory units preserves ~83% of internal dependencies versus 68% for matched nonfunctional groups, and outer typed relations raise cross-experience support coverage from 52% to 69% (Fig. 1b).
- The value of a memory composition depends on its presentation format, so feedback must record composition-format pairings jointly; joint updates beat separate scoring (85% vs 75% probe success at 192 calibration tasks, Fig. 1d).
- MACE achieves the best score on all eight benchmarks, averaging 81.11% versus 78.97% for the strongest baseline SAGE and 71.53% for direct answering (Table 1).
- Every component is necessary: removing the Coupler (−5.88 pp), MemGoG (−5.20 pp), Composer (−3.53 pp), or Feedback Co-Evolver (−1.41 pp) each lowers the average (Table 2).
Strengths
Evidence-first design motivation
Sec. 4's four diagnostics isolate variables with unusual care for this literature: 'Matched' controls equalize group count, size, overlap, and connectivity with functional units (App. A), budgets are fixed at 256-2,048 tokens, and dependency-retention denominators are pre-annotated requirement sets (Fig. 1a-b: 83% vs 68% internal retention; 52%→69% coverage).
Composition-format interaction is a genuine finding
Fig. 1c shows the preferred unit combination reverses between guide and checklist formats (S1: 82% vs 76% as guide, 59% vs 78% as checklist) with content held fixed — a non-obvious result that directly justifies pairing-level feedback (Fig. 1d: Joint 85% vs Separate 75% at 192 calibration tasks).
Unusually thorough protocol documentation
Broad, multi-axis evaluation
Competitive efficiency profile
Weaknesses
Single-run statistics with hair-thin margins
Uncited and protocol-mixed baselines
Cross-table inconsistencies
The titular mechanism contributes least
Unnamed backbone and no code release
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
MACE (arXiv 2609.21533, cs.LG) targets experience reuse in LLM-based multi-agent systems. The authors argue that reusable procedures must preserve dependencies (conditions → actions → outputs) and contribute: (1) four empirical studies motivating "functional memory units" — subgraphs grouping these dependencies; (2) MemGoG, a graph-of-graphs memory whose units are linked by typed relations (support, conflict, repair, abstraction, operationalization, role projection, feedback; §5.2); (3) the MACE Loop — a Need-aware Memory Composer (budgeted selection, Eq. 6-7), an Adaptive Agent-Memory Coupler (role-specific patches rendered as instructions or checklists, Eq. 9-10), and a Feedback Co-Evolver (utility updates, Eq. 12). Headline claim: best score on all eight benchmarks, 81.11% average vs 78.97% for the strongest baseline SAGE, +13.40% relative over Direct (Table 1).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is MACE and what problem does it solve?
MACE is a framework that helps LLM-based multi-agent systems reuse past collaboration experience. It stores previous task traces as functional memory units — subgraphs linking conditions, actions, and outputs — connected by typed relations (support, conflict, repair). At query time it selects a task-relevant subset under a token budget, delivers it to each agent as instructions or checklists, and updates unit scores and relations from task outcomes.
How much better is MACE than existing methods?
In the paper's single-run evaluation over eight benchmarks (4,728 queries), MACE averages 81.11%, versus 78.97% for the strongest compared baseline (SAGE) and 71.53% for direct LLM answering. Margins on some benchmarks are very small (e.g., 0.08 points on GSM8K), and no statistical significance testing is reported, so the size of the true advantage is uncertain.
What are MACE's main limitations?
All results come from single runs without variance estimates; several winning margins are within noise; key baselines (SAGE, MAGMA, GraphSearch, R-GFM) are uncited and some numbers are imported under different evaluation protocols; the LLM backbone used is never named; the paper's own ablations show the feedback/co-evolution component contributes the least (−1.41 points); and robustness tables contain values inconsistent with the main results table.
Is code or data available for MACE?
The reviewed text describes extensive logging, fixed sampling seeds, and reproducibility appendices, but provides no code repository, model identity, or public data release. Independent verification currently requires a full reimplementation, including the memory-graph construction pipeline.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 appear only as garbled axis-label text; plot-level values were taken from the prose and Appendix C.5 tables, and curve shapes could not be verified.
- The serving LLM backbone, temperature, and several hyperparameters (e.g., η in Eq. 12, budget values) are absent from the extracted text; they may exist in truncated portions or may genuinely be unspecified.
- The reference list contains 11 entries; MAGMA, SAGE, GraphSearch, and R-GFM have no entries at all — treated as genuinely missing rather than extraction loss.
- Table 20/21 cells for shared settings beyond sampling rules appear sparse in the extraction; some hyperparameter values may be lost to truncation.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.