An Open Recipe for IMO Gold: Training Nemotron for Olympiad Mathematics
Ivan Moshkov, Stephen Ge, George Armstrong, Wei Du, Sadegh Mahdavi, Igor Gitman
cs.AI
Quick verdict
Claims an open NL pipeline hit 30/42 gold at IMO 2026; transparent, well-ablated system report, but model-jury blind spots and n=30 dev-set ablations temper it.
Summary
NVIDIA researchers post-trained two specialist checkpoints (supervised fine-tuning and reinforcement learning) from Nemotron 3 Ultra (550B-A55B MoE) for proof generation, verification, and refinement, then combined them with the general-availability checkpoint in a fully natural-language, tool-free pipeline entered officially at IMO 2026. It scored 30/42 under official grading — full credit on Problems 1, 2, 4, and 5, one point each on Problems 3 and 6 — above the 29-point gold cutoff. The SFT stage used a 414,890-example synthetic corpus generated with DeepSeek-V4-Pro; RL followed DeepSeekMath-V2's reward design on 9,597 problems. The submitted system runs up to eight generate-verify-refine rounds (acceptance requires unanimous 16-judgment panels) followed by 48-judgment final selection. On a 30-problem development set, both specialists beat the GA checkpoint (fallback jury totals 180/162/165) and the ensemble leads at 188; a 300-proof verifier audit with clustered bootstrap CIs puts the deployed RL+SFT unanimity rule at 1.1% false accepts and 81.3% false rejects. Key limitations, largely self-reported: internal verifier and independent model jury both overestimated the official score by about two points (a shared P3/P6 blind spot), ablation differences lack significance testing on the small dev set, and the recipe depends on external frontier models for data generation and judging. Released: checkpoints, data, code, submitted proofs, compute accounting, and a 200-problem benchmark.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A fully natural-language system with no formal prover, tools, or internet scored 30/42 at IMO 2026 under official grading, above the 29-point gold cutoff.
- Both post-trained checkpoints beat the GA checkpoint in complete single-model pipelines, and the three-checkpoint ensemble beats the best single checkpoint (188 vs 180 fallback jury points on 30 dev problems).
- Requiring unanimity of the two selective verifiers (RL+SFT, 16 judgments) reduces false accepts to 1.1% at 81.3% false rejects, and this asymmetric operating point is the right one because a false accept terminates search.
- Spending round-1 compute across checkpoints beats doubling one checkpoint's attempts (RL 128+SFT 128: 18 accepted problems vs RL 256: 14).
- Gains came from complementary checkpoints, verification-guided refinement, and high-compute final selection; elaborate routing and refinement variants (triage, cross-proof context, diverse-proof synthesis) did not improve final coverage.
Strengths
Official, verifiable competition result with exemplary resource accounting
Scored 30/42 under official IMO 2026 grading (gold cutoff 29); Table 1 breaks down per-problem tokens and GB200 GPU-hours — 707M tokens / 1,464 GPU-h to the six submitted proofs, 2.31B tokens / 4,800 GPU-h for the full run.
Unusually complete open release
§2 enumerates both post-trained checkpoints, the SFT corpus and RL problem set, NeMo-Skills/NeMo-RL code, the submitted proofs, exact prompts (Appendix B, Listings 1–18), and Nemotron-IMO-Bench (200 novel problems, CC BY 4.0), plus a script assembling the 30-problem dev set.
Careful, decision-relevant verifier evaluation
Honest negative results and error analysis
Attempt-budget ablation directly supports the ensemble design
Weaknesses
All evaluation ground truth is model-based and demonstrably optimistic
Small development set with no uncertainty quantification outside Table 3
The open recipe depends on external frontier models
Compute barrier to practical reproduction
Dev-set provenance only partially decontaminated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This NVIDIA technical report describes the system entered officially at IMO 2026: three Nemotron 3 Ultra checkpoints (550B-A55B MoE) — the general-availability model plus SFT and RL specialists — driving a purely natural-language generate-verify-refine pipeline with no formal prover, tools, or internet. The headline claim is 30/42 under official grading (gold cutoff 29), with full marks on Problems 1, 2, 4, and 5 and one point each on Problems 3 and 6. Claims are unusually well scoped: 'accepted' is explicitly defined as an internal verification criterion (§5.1), and the post-cutoff improvement to a putative 33 points is flagged as an unofficial human regrade (§5.3, Table 1). A large artifact release (§2) backs the 'open recipe' framing.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What score did the Nemotron system get at IMO 2026?
30 out of 42 points under official IMO grading — full credit on Problems 1, 2, 4, and 5 and one point each on Problems 3 and 6 — above the 29-point gold-medal cutoff. The system worked entirely in natural language, with no formal prover, external tools, or internet access.
Is the Nemotron IMO system open source?
Largely yes: the two post-trained checkpoints (SFT and RL), the SFT corpus and RL problem set, training/inference code, the submitted proofs, compute accounting, and a 200-problem benchmark (Nemotron-IMO-Bench) are released via Hugging Face and GitHub. However, the original data pipeline relied on external models (DeepSeek-V4-Pro for generation, GPT-5.5 for RL evaluation), so the full recipe cannot be re-executed without those services.
How much compute did the IMO 2026 run use?
About 707M generated tokens and 1,464 GB200 GPU-hours to produce the six submitted proofs, and roughly 2.31B tokens and 4,800 GPU-hours for the full competition run including rounds already in flight. A post-cutoff continued run on Problem 6 added ~890 GPU-hours.
Why did the model verifiers disagree with the official IMO result?
Both the internal search verifier and an independent model jury credited about 32/42, two points above the official 30. The gap comes entirely from Problems 3 and 6, where the model judges gave more credit than the official graders — the authors attribute this to a shared blind spot in model-based proof verification rather than noise in one component.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 and 2 appear in the extraction only as scattered axis labels plus caption text, so curve shapes and timing details rest on the captions and Table 1 rather than the plotted curves.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.