Learning What to Retain: Gated-Memory Routing for Efficient Collaboration in Multi-Agent LLM Systems
Rakibul Hasan Rajib, Mengxin Zheng, Qian Lou
cs.AI · cs.CL
Quick verdict
Gated-memory routing: claims full-history accuracy parity at far lower cost, tops baselines by 2.44 avg points; cost metric self-defined, margins untested.
Summary
The paper recasts multi-agent LLM orchestration as routing over a learned, selective execution memory instead of conditioning decisions on the query alone (MASRouter-style) or the full trajectory (Puppeteer-style). A jointly trained policy stack — a VAE-embedded role allocator, a role-conditioned backbone router, a stochastic Memory Write Gate scored by a learned MMR relevance-novelty trade-off, a per-record Retrieval Gate, and a GRU-based Adaptive Halting Controller — is optimized end-to-end with group-relative (GRPO-style) policy gradients under a cost-weighted binary-success reward, over five open-weight backbones (3B-32B). On MATH, GSM-Hard, MBPP, HumanEval, and MMLU-Pro it reports the best average accuracy (77.73), +2.44 over the strongest baseline (Puppeteer-32B), cutting HumanEval cost 31.9% versus that baseline (43.9% vs MASRouter); MBPP is the exception, where a fixed MacNet chain wins. Ablations (Table 2) attribute accuracy mainly to backbone routing (-14 GSM-Hard points without it) and cost control to halting; Table 3 shows gated memory matches full-history accuracy at roughly 40-50% lower cost. Three-seed variance (Table 8) is reported with stable rankings. Limitations: the cost metric is a self-defined parameter-count proxy, no significance tests are reported, the GSM-Hard cost exceeds Puppeteer-32B despite the figure caption, and evaluation covers only verifiable closed-domain tasks with models up to 32B.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 4
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Conditioning routing on a learned gated execution memory achieves the best average accuracy (77.73) across MATH, GSM-Hard, MBPP, HumanEval, and MMLU-Pro, exceeding the strongest baseline (Puppeteer-32B, 75.29) by 2.44 points (Table 1).
- Gated-memory routing matches full-history routing accuracy while costing about 40% less on GSM-Hard and over 50% less on HumanEval (Table 3).
- Learned write and retrieval gates keep the memory compact and non-degenerate, each contributing accuracy in leave-one-out ablations without collapsing to trivial policies (Table 2, Table 9, Appendix 12).
- The Adaptive Halting Controller is the primary cost lever: removing it raises test cost by over 40% on GSM-Hard and more than doubles it on HumanEval, while also cutting training rollout steps by ~1.8× (Section 5.3, Appendix 13).
Strengths
Direct paradigm isolation
Section 5.5 (Table 3) holds the routing stack fixed and swaps only the state representation: gated memory reaches 70.55 vs 70.27 (GSM-Hard) and 89.84 vs 89.06 (HumanEval) of full-history routing at 40.4% and 52.9% lower cost — the cleanest test of the core claim in the paper.
Budget-matched, broad baselines
Section 4.1 equalizes agent budget (all multi-agent baselines at max depth φ=6) and reuses the identical 26-role MASRouter catalog, covering single models, CoT variants, MacNet topologies, AFlow, Puppeteer, and MASRouter (Table 1).
Ablations match the claimed division of labor
Variance and degeneracy checks are reported
Cost accounting is documented and partially corroborated
Weaknesses
Headline efficiency rests on a self-defined cost metric
Cost win over the strongest baseline is benchmark-specific, and a figure caption contradicts Table 6
Accuracy contribution is dominated by backbone routing, not memory gating
No statistical testing; operating-point selection undisclosed
Narrow evaluation envelope
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper proposes Gated-Memory Routing (GMR): multi-agent LLM orchestration decisions — role, backbone, context, and stopping — are conditioned on a learned, gated execution memory rather than the query alone or the raw trajectory (Sections 1, 3). Components: a History-Aware Role Allocator (VAE latents over 26 textual role profiles, §3.3), a role-conditioned LLM Router over five open-weight backbones 3B–32B (§3.4), a stochastic per-record Retrieval Gate (§3.5), a Memory Write Gate scored by a learned MMR relevance–novelty trade-off (§3.6), and a GRU Adaptive Halting Controller with max depth φ=6 (§3.7), all trained end-to-end with GRPO-style group-relative policy gradients under binary success minus size-proportional cost (§3.8). Central claims: best average accuracy 77.73 across MATH/GSM-Hard/MBPP/HumanEval/MMLU-Pro, +2.44 over Puppeteer-32B (Table 1); 31.9% HumanEval cost reduction vs that baseline; and full-history accuracy parity at ~40–50% lower cost (Table 3).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Gated-Memory Routing in one sentence?
It is a framework in which a multi-agent LLM system makes every orchestration decision — which agent role acts, which LLM backbone powers it, what context it reads, and when to stop — conditioned on a learned, selectively filtered execution memory rather than on the raw query alone or the full unfiltered history of prior steps.
How much does it improve over prior methods?
On five benchmarks (MATH, GSM-Hard, MBPP, HumanEval, MMLU-Pro), the paper reports the best average accuracy of 77.73%, which is 2.44 points above the strongest baseline (Puppeteer-32B at 75.29%), while claiming 31.9% lower inference cost on HumanEval than that baseline under the paper's own parameter-count-based cost model. MBPP is the one benchmark where a fixed multi-agent chain (MacNet on qwen-2.5-14B) scores higher.
Is code or data available for this paper?
The abstract states code is available at github.com/rajibrhasan/gated-memory-routing. The appendices are unusually detailed, providing the algorithm pseudocode, all aggregator prompts, the 26 agent role descriptions, backbone capability profiles, per-model cost rates, hyperparameters, and a three-seed variance protocol; all five benchmarks and backbones are public.
What are the main limitations of this work?
Evaluation is restricted to verifiable closed-domain tasks (math, code, multiple-choice QA) with open-weight models up to 32B; the cost metric is a self-defined proxy based on parameter counts, with hardware measurements on only two benchmarks; no statistical significance tests are reported; and on GSM-Hard the method's own cost table shows it is more expensive than the strongest baseline, despite a figure caption suggesting otherwise.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-5 are available only as extracted axis text in this extraction, so exact per-baseline Pareto coordinates in Figures 3-5 could not be verified beyond the percentages quoted in the prose and the appendix cost tables.
- The GitHub repository linked in the abstract could not be inspected (offline review), so the code-release claim is taken as stated.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.