Stellar Colosseum: A Many-Agent Harness for Long-Horizon Research in Mathematics and Theoretical Computer Science
Honghao Lin, David P. Woodruff, Yuan Deng, Jieming Mao, Song Zuo, Vahab Mirrokni
cs.AI · cs.CL · cs.LG
Quick verdict
Stellar Colosseum proposes a staged, critique-preserving multi-agent harness for long-horizon math/TCS research and reports 71.0% on TCS-Bench and 218/222 Codeforces, but the evidence rests on proprietary models, an automated grader, and compute-unmatched baselines.
Summary
Stellar Colosseum is a model-agnostic harness that allocates inference across long-horizon research in mathematics and theoretical computer science. At the workflow level (Section 4.2) it runs five stages: parallel strategy exploration, a readiness gate that decides whether a route is concrete enough to decompose, decomposition into a sectioned proof skeleton with an explicit dependency DAG, parallel subproblem solving with local retry, and global verification that can trigger revision or re-exploration. Within each stage (Section 4.1) it samples a population of typed candidates, attaches targeted falsification reports to each, and reduces them through an overlapping random-sample tree whose expected reuse is m_{l+1}*k_l/m_l (about 2.5 for a 128-to-64 transition with sample size 5); critiques ride along with the proposals they attack rather than being averaged away. A knowledge directory preserves lemmas, failed approaches, references and observations across rounds. The paper reports five new research results obtained with the workflow (Section 5): improved strong coresets for l_p subspace approximation at p>2 (size tilde-O(k^{p/2} eps^{-2}) instead of eps^{-p}), a condition-number barrier for sparse least squares under the randomized exact-volume Small-Set Expansion Hypothesis, near-optimal dimension lower bounds for single-vector embeddings of maximum inner product similarity, a single-stage Hadamard quantization estimator removing the residual stage and cutting the leading constant by about 5.93x, and a near-optimal gamma_{2,1}(Q) = Omega(log^{3/2} n / (log log n)^{3/2}) lower bound for prefix-matrix factorizations. Case studies cover 46-page and 75-page Knuth cycle proof drafts and a 22-page, 15-round independent rediscovery of the Erdos unit-distance architecture. On TCS-Bench (300 tasks from FOCS/STOC/SODA 2020-2026), Colosseum with Gemini 3.1 Pro scores 54.0% and with Gemini 3.7 Flash 55.0%; cross-model critique selection reaches 71.0% versus 30.3% for direct Gemini 3.1 Pro and 68.0% for GPT-5.6 Pro (max). On 222 Codeforces problems, the execution-probe configuration records 218 accepted solutions (rating 4263) versus 213 (3918) without it. Main limitations: no component ablations, no compute-matched baselines, reliance on an automated reference-assisted grader, and unreleased code or models.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Long-horizon mathematical research can be organised as an explicit pipeline separating strategy exploration, readiness gating, DAG-structured proof decomposition, parallel subproblem solving and global verification, with revision and re-exploration as first-class return paths.
- Within any stage, generating a diverse candidate population, attaching targeted falsification reports, and reducing them through an overlapping random-sample tree that preserves objections outperforms flat voting because substantive disagreement and falsification evidence survive aggregation.
- Running Colosseum on TCS-Bench with two different base models and using cross-model critique to select between the two proofs reaches 71.0% accuracy, above either single run (54.0%, 55.0%) and above direct GPT-5.6 Pro (max) at 68.0%.
- The same proof-oriented architecture transfers to competitive programming: with a C++ execution probe added to the verification-revision loop, 218 of 222 Codeforces problems are accepted (rating 4263) versus 213 (3918) without the probe.
- Runs of the workflow contributed to five new results addressing open problems from FOCS and JMLR papers, including improved l_p subspace coreset size, a condition-number barrier for sparse least squares, and near-optimal dimension lower bounds for maximum-inner-product embeddings.
Strengths
Explicit, mechanically specified aggregation operator
Section 4.1.3 gives the sampling distribution G ~ Unif{subsets of size k}, the width schedule (m_0..m_L), and the closed-form expected reuse E[R] = m_{l+1}*k_l/m_l with the 128-to-64/k=5 example giving 2.5. This is unusually concrete for an agent-architecture paper and makes the overlap-vs-partition distinction (groups overlap, do not partition) reproducible at the design level.
Critiques are preserved through aggregation rather than voted away
The root artifact returns 'a synthesized candidate together with any unresolved objections' (Section 4.1.3), and the global verifier states that 'a concrete fatal defect is sufficient to reject the proof' (Section 4.2.3). This directly addresses the known failure mode of self-consistency, where agreement among samples can hide a shared error, which the paper itself names in Section 1.
Dependency-aware decomposition with localised repair
Large, well-characterised benchmark evaluations
Honest handling of partial progress and failure
Weaknesses
No component-level ablation of the architecture
Baselines are not compute-matched
All benchmark accuracies depend on an automated grader with unvalidated error modes
The five research results are asserted, not established, in this paper
Reproducibility is effectively zero for external readers
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Stellar Colosseum is a systems paper, and it is clearest when read as one. The authors state four contributions in Section 1: (i) formulate long-horizon research as a pipeline separating strategy exploration, readiness gating, proof decomposition, subproblem solving and global verification; (ii) develop a stage-level adversarial inference procedure combining diverse generation, targeted falsification and overlapping tree-structured aggregation; (iii) demonstrate the workflow on open-ended problems with Gemini 3.1 Pro; (iv) evaluate it on TCS-Bench and competitive programming. The word "model-agnostic" is load-bearing: the architectural claim is meant to be separable from any particular base model.
The scope boundary that matters most is Section 5. Five new mathematical results are summarised there — improved strong coresets for l_p subspace approximation at p>2, a condition-number barrier for sparse least squares, near-optimal dimension lower bounds for maximum-inner-product embeddings, a single-stage Hadamard quantization estimator, and a near-optimal gamma_{2,1}(Q) lower bound for prefix-matrix factorizations — but every one of them defers definitions, attribution and proofs to companion manuscripts [38-41]. The paper therefore claims credit for producing mathematics it does not establish. That is a defensible division of labour, but it means the correctness of the headline research results is outside what this document can support.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Stellar Colosseum and what problem does it solve?
Stellar Colosseum is a model-agnostic multi-agent harness from Google Research for long-horizon research in mathematics and theoretical computer science. Instead of asking one model to write a whole proof in a single response, it splits the work into strategy exploration, a readiness gate, decomposition into a dependency graph of section-level subproblems, parallel solving with local retries, and global verification. Within each stage it samples many candidates, has adversarial reviewers attack each one, and merges them with an overlapping random-sample tree that keeps the criticisms attached to the proposals they target.
How well does Colosseum perform on TCS-Bench?
On TCS-Bench, a set of 300 research-level theorem-proving tasks drawn from FOCS, STOC and SODA papers published 2020-2026, Colosseum with Gemini 3.1 Pro scores 54.0% and with Gemini 3.7 Flash 55.0%. Using one model's critics to choose between the two candidate proofs raises this to 71.0%. Direct baselines are 30.3% for Gemini 3.1 Pro, 52.0% for Gemini 3.1 DeepThink and 68.0% for GPT-5.6 Pro (max). An oracle that always picks a correct proof when one exists would reach 77.3%. All scores come from an automated reference-assisted grader reporting above 90% accuracy on 100 expert-labelled proofs.
Is Colosseum's code or the underlying model publicly available?
No. The paper releases neither the harness code nor the base models (Gemini 3.1 Pro, Gemini 3.7 Flash), and Appendix A states that the prompt templates shown are shortened versions with implementation details omitted. Only some generated proof artifacts are public, for example the Knuth cycles repository and the 22-page Erdős unit-distance draft on GitHub. External reproduction of the reported numbers is therefore not currently possible.
What new mathematics did Colosseum actually produce?
The paper summarises five results that runs of the workflow contributed to, with full proofs deferred to companion manuscripts: a strong coreset of size Õ(k^{p/2} eps^{-2}) for l_p subspace approximation at p>2 improving on eps^{-p}; a conditional barrier showing sparse least squares cannot avoid linear condition-number dependence; near-optimal dimension lower bounds for single-vector embeddings of maximum inner product similarity; a single-stage Hadamard quantizer whose leading constant is about 5.93 times smaller; and a near-optimal gamma_{2,1} lower bound for prefix-matrix factorizations. Because the proofs live in the cited papers, correctness cannot be checked from this paper alone.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 (the architecture diagram) and the tables describing tree configurations are only partially recoverable from the extracted text; the Table 1 and Table 2-5 values quoted here come from the flattened table text and should be checked against the rendered PDF.
- Mathematical statements in Section 5 are reproduced from the extracted text, where sub/superscripts and tilde-O notation are flattened; readers should consult companion papers [38-41] before quoting any bound.
- The five research results and the two case studies cannot be verified from this document because their proofs are external; judgments about their correctness are necessarily out of scope.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.