Do We Need Complex Topology Control? Distinct-Peer Random Routing Improves Cost-Efficiency in Sparse Multi-Agent Debate
Boxuan Wang, Zhuoyun Li, Xiaowei Huang, Yi Dong
cs.AI
Quick verdict
Argues random distinct-peer routing + round-3 stopping rival complex MAD topology control; cost story solid, accuracy gains within seed noise, no code.
Summary
The paper asks whether sophisticated topology control in sparse multi-agent debate (MAD) is necessary, and answers largely in the negative. The authors propose a two-axis evaluation framework separating peer routing (who communicates) from stopping (how long), using eight homogeneous GPT-4o-mini agents that each receive two peers' answers per round across five benchmarks (ARC-Challenge, ScienceQA-text, GSM8K, MMLU-Pro, GPQA-Diamond). The proposed Random-NoDup policy resamples two distinct peers each round: at full 14 rounds it slightly exceeds the Ring baseline's macro accuracy (76.87% vs 76.27%) at similar token cost and outscores stage-based/random/uncertainty-guided switching and an AgentPrune-style learned adaptation. A short variant stopping at round 3 achieves the best macro accuracy (77.33%) at about 12.4k tokens per question versus roughly 57k for full debate — about a 78% reduction. Cross-model checks on three tasks with GPT-4.1-mini, GPT-4.1, and GPT-5.6 Luna confirm token savings for every model-task pair, while accuracy effects vary (GPT-4.1 loses about 5 points on GSM8K with early stopping). Main limitations: accuracy gaps are under one point, below seed variability (up to about ±5), no significance tests or per-benchmark sample sizes are reported, the learned baseline is the authors' own reimplementation, and no code or prompts are released. The efficiency conclusion is convincing; the accuracy-equivalence claims rest on statistically fragile evidence.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A random-without-replacement policy sampling two distinct peers per round (Random-NoDup) matches or slightly exceeds structured, adaptive, and learned topology control at matched token cost in sparse MAD (Table 1: macro 76.87% vs 76.27% for Ring).
- Stopping Random-NoDup at round 3 yields the best observed macro accuracy (77.33%) while cutting token cost from about 57.4k to 12.36k per question, roughly a 78% reduction (Table 1).
- Token savings from early stopping transfer across all twelve tested model-task pairs over four models, whereas accuracy effects are model- and task-dependent (Table 2; GPT-4.1 drops about 5 points on GSM8K).
- Learned or adaptive topology-control methods should be benchmarked against strong simple routing and stopping baselines before their added complexity is justified.
Strengths
Cleanly matched experimental protocol
All methods share eight homogeneous agents, received degree k=2, synchronous updates, identical prompts/generation settings and the 14-round horizon (Sec 4.1, Table 1). Separating routing from stopping (Fig 1b, Sec 3.2) is a genuinely useful evaluation design that attributes effects to the right axis.
Honest statistical labeling
Table 1's caption explicitly states bold marks best observed mean, not significance; Sec 4.3 reports paired intervals including zero for round 2-5 cutoffs; the conclusion scopes claims to no detected advantage in tested comparisons rather than universal superiority.
Practically large and robust efficiency result
Includes a learned comparator under a matched protocol
Weaknesses
Statistically fragile accuracy claims
The efficiency win is close to built-in
Learned baselines are adaptations, not original systems
Reproducibility gaps
Benchmark saturation and odd reference list
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper addresses a narrow, well-posed question: in sparse multi-agent debate (MAD) over homogeneous LLM agents, does sophisticated topology control (learned, adaptive, or scheduled routing) improve the accuracy-cost trade-off over simple random routing, and does stopping control help further? The authors propose Random-NoDup — each agent samples two distinct peers uniformly every round — and a two-axis framework separating routing from stopping. Central claims: (1) Random-NoDup matches or slightly exceeds all tested routing controllers at full rounds; (2) stopping at round 3 gives the best macro accuracy (77.33%) at ~12.4k tokens/question versus ~57k for full debate; (3) complex topology control should be benchmarked against such simple baselines.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Random-NoDup routing in multi-agent debate?
In each communication round, every agent uniformly and randomly draws two distinct peers (without replacement within a round; peers may repeat across rounds) and reads only their previous-round answers. It requires no learning, no state tracking, and no task heuristics.
Does multi-agent debate need complex communication topologies?
According to this paper, not necessarily. Under a matched sparse protocol (8 GPT-4o-mini agents, 2 peers each, 5 benchmarks), random distinct-peer routing slightly beat ring, switching policies, and an AgentPrune-style learned topology at similar cost, and the reliable gain came from stopping early at round 3, which cut tokens about 78% with roughly preserved accuracy — though accuracy effects vary by model.
How much cheaper is short (round-3) debate than full 14-round debate?
On the five-benchmark macro average in Table 1 (GPT-4o-mini), about 12,360 tokens per question versus 57,380 — roughly 78% fewer. Macro accuracy was 77.33% for the short version versus 76.87% for full debate, but in cross-model tests (Table 2) one model-task pair (GPT-4.1 on GSM8K) lost about 5 accuracy points from early stopping.
Is code or data available for this paper?
The paper text announces no code, prompt, or data release. The benchmarks used (ARC-Challenge, ScienceQA-text, GSM8K, MMLU-Pro, GPQA-Diamond) are public, but exact evaluation subset sizes are not stated, so exact-number reproduction is currently not possible without the authors.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1(c) and 3 are only available as extracted text descriptions; the per-round accuracy-cost curves and shaded regions could not be inspected visually.
- Benchmark question counts are stated nowhere in the extracted text, limiting statistical interpretation of the reported SDs.
- Table 2 short runs are same-trajectory prefixes while Table 1 C1 uses independent runs; the numbers across tables are not directly comparable.
- References [23]-[30] are uncited in the visible body text, so the extracted version may differ from the final PDF.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.