Self-Organizing Agent Teams Learn to Reason Together
Aneesh Pappu, Mirac Suzgun, Yongchan Kwon, Federico Bianchi, Batu El, Mykel J. Kochenderfer, Hancheng Cao, James Zou
cs.AI · cs.MA
Quick verdict
SAT learns teamwork strategies that beat best-member, matched compute, and a perfect router on math-physics, though per-seed variance is unreported.
Summary
Self-Organizing Agent Teams (SAT) are fixed three-model teams that learn reusable teamwork strategies offline via evolutionary search: the strongest member reflects on past collaborations and mutates roles, phases, and information-flow rules, learning a frozen ten-strategy bank from just 15 AIME-2024 (or 25 GPQA Diamond) problems. At test time the bank yields candidate certificates per problem and a member judge audits and selects. The math-physics team averages 66.7% over five benchmarks versus 48.8% for the strongest member, 58.7% for compute-matched linearization, and 59.0% for a perfect router over members' independent answers; on AIME 2026 it exceeds the router by 13.4 points. Mixture-of-Agents and the homogeneous team still win on some benchmarks (e.g., MMLU-Pro, BBEH). The knowledge-and-logic team is best on average (72.8%) yet below the 79.6% routing oracle, exposing selection as a bottleneck. Gains rank-correlate with demonstrability (rho=0.90) across eight benchmarks, post hoc. Results are three-seed means on proprietary models without variance, search costs are undisclosed, and no code is released.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Fixed teams of LLM agents can learn reusable teamwork strategies (roles, phases, participation, information flow) offline from as few as 15-25 training problems, and these strategies transfer unchanged to unseen problems and benchmarks.
- Learned organization exceeds compute-matched single-agent inference and a perfect router over members' independent answers on a five-benchmark math-physics suite (66.7% vs 58.7% and 59.0%), showing collaborative computation creates correct solutions no member produced independently.
- On knowledge-and-logic tasks the team leads on average accuracy (72.8%) but stays below the 79.6% routing oracle despite 87.9% coverage, separating solution generation from solution selection.
- Demonstrability — whether correct reasoning can be recognized once produced — strongly tracks the team's improvement over its strongest member across eight benchmarks (Spearman rho = 0.90, p = 0.005).
Strengths
Stricter-than-usual evaluation target
The routing oracle — a perfect per-problem selector over members' independent answers, explicitly framed as the computational analogue of the truth-wins condition (Laughlin & Ellis, 1986) — is a legitimately demanding bar, and the math-physics team clears it on average (66.7% vs 59.0%, Table 1), with a case (HMMT 2026 #6, Figure 5/9) where all three initial answers are wrong yet the composed answer is correct.
Comprehensive control battery
Beyond best-member, debate, Mixture-of-Agents, member-vote, and self-consistency (K=10), the paper includes compute-matched linearization (58.7% vs SAT's 66.7%) and a homogeneous-team control (56.0%) that isolates model heterogeneity (Table 1); knowledge-suite linearization at 72.1% vs SAT 72.8% honestly shows a much thinner margin there (Table 2).
Transfer with frozen artifacts
Generation vs selection decomposition
Weaknesses
No variance or significance reporting
Learning cost never quantified
Judge and reflector are the same, strongest member
Mixed per-benchmark results sit inside an averaged headline
Strategies encode roster-specific evidence
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Review: Self-Organizing Agent Teams Learn to Reason Together (arXiv:2609.22682)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is SAT (Self-Organizing Agent Teams) and how does it work?
SAT is a method where a fixed team of three AI models learns reusable 'teamwork strategies' — natural-language instructions specifying roles, conversation phases, participation, and information flow — through an offline evolutionary search on a tiny training set (15-25 problems). The finished strategy bank is frozen, then run on every new problem to produce candidate solutions, and one member model audits the written certificates and picks the final answer. No model weights are trained; only the organizational prompts are learned.
How much does SAT improve over baselines and on which benchmarks?
On five math and physics benchmarks, SAT averages 66.7% accuracy versus 48.8% for its strongest single member, 58.7% for compute-matched single-agent inference, and 59.0% for a perfect router selecting among members' independent answers; on AIME 2026 it beats that router by 13.4 points. Gains are not universal: Mixture-of-Agents wins on MMLU-Pro and a homogeneous team wins on BBEH. On knowledge/logic tasks SAT is best on average (72.8%) but below the 79.6% routing ceiling.
What is 'demonstrability' and why does it matter in this paper?
Demonstrability is an organizational-psychology idea: whether a team can tell correct reasoning from incorrect reasoning once it appears. The paper measures it with a ten-small-model panel that judges team reasoning, and finds benchmarks where correct reasoning is easier to recognize are exactly where the team improves most over its strongest member (Spearman rho = 0.90 across eight benchmarks). The authors stress this is a correlational, post-hoc observation.
Is code or data available, and can I reproduce the results?
The paper publishes the complete learned strategy prompts (Appendix A) and judge templates (Appendix D), and all benchmarks are public. However, no code or strategy-bank artifact release is announced, results depend on six proprietary APIs (o3-mini, Claude Sonnet 4, DeepSeek-V3, Gemini-2.5-Flash, Llama-4-Maverick, GPT-4.1), and no per-seed variance is reported, so exact reproduction requires paid API access and would be noisy.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Review is based on extracted text; several figures (1-4, 7, 8) arrived as garbled diagram fragments and some case-study transcripts were truncated, so visual/qualitative claims rest on the condensed text.
- Appendices appear complete in the extraction (strategy banks, judge prompts, demonstrability tables), so the audit of methods and statistics is text-grounded; no external checking was performed.
- 'Compute-matched' applies to the linearization inference control only; whether team search and deployment are cost-competitive overall cannot be assessed from the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.