Learning to Discover Interesting Mathematics
Niket Patel, Ahmad Rammal, Amaury Hayat, Remi Munos, Julia Kempe
cs.LG · cs.AI
Quick verdict
Claims proof-difficulty÷statement-length is a useful 'interestingness' objective steering LLMs to interesting, novel Lean theorems and recursive discovery; evidence promising but judge-based and variance-free.
Summary
The paper operationalizes mathematical interestingness for Lean 4 as conditional proof difficulty divided by description length. From mathlib's dependency graph, premise expansion yields ~100k training points; a Qwen3.6-27B trained with GRPO (truth, Bellman, and monotonicity reward terms) predicts proof lines more accurately than GPT-5.5 and Claude Opus 4.6. Interestingness is defined as difficulty over description length; utility as downstream uses times difficulty; they correlate strongly (Spearman ρ=0.756 after excluding zero-user theorems). Using interestingness as GRPO reward trains a conjecturer producing statements with harder proofs relative to statement length: over 160 verified statements, mean interestingness rises from 1.76 to 7.58 in all eight areas, and substantial-or-full mathlib overlap falls from 91.9% to 30.6%. In an inference-time ablation, interestingness-based promotion is ranked first in 60% of blinded judge comparisons. Limitations: dependence on Lean proof style, small samples without reported variance, an LLM judge sharing a model family with one baseline, and unproven future utility.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Conditional proof difficulty V(T|P), measured as Lean 4 proof lines, can be learned by a 27B GRPO-trained model more accurately than by frontier general-purpose models (Figure 1).
- Intrinsic interestingness — proof difficulty divided by conditional description length — correlates strongly with extrinsic downstream utility (Spearman ρ=0.756 excluding zero-user theorems, Figure 3).
- RL training on the interestingness reward quadruples verified mean interestingness (1.76→7.58, Figure 4) while cutting substantial-or-full mathlib overlap from 91.9% to 30.6% (Figure 5).
- Inference-time selection by interestingness is the best promotion rule for iterative, self-expanding discovery (60% ranked-first in blinded comparisons, Tables 1-3), beating raw proof-length pruning.
Strengths
A computable, formal-library-grounded interestingness proxy
Section 3.1 defines I(T|P)=100·V(T|P)/L(T|P), where V is conditional proof lines in Lean 4 and L counts premise-relative definition description length (Eq. 5-6). Figure 2 validates it on mathlib: trivial identities like 1^n=1 rank at the bottom, Fermat's Last Theorem (n=3) at the top — the metric's ordering matches mathematical intuition.
Careful value-function design and leakage control
Definition 1's axioms (grounding, premise monotonicity, composition) directly shape the GRPO reward (Ltruth, LBellman, Ldrop, Appendix B.2/Algorithm 1); the six-role prompt construction (Appendix B.1.1: base, remaining_keep, remaining_remove, lemma_context, lemma_native, base_drop) is a clean way to manufacture supervision, and splits are by root theorem so variants never straddle train/validation.
Joint evidence that gains are not library regurgitation
Well-isolated inference-time ablation
Weaknesses
Interestingness is a proof-engineering artifact as much as a mathematical property
No dispersion or significance for any headline number
Judge circularity
Utility proxy is definitional, and the correlation excludes the nulls it measures
No release statement and small-scale downstream evidence
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks how a self-improving mathematical system should decide *what to prove*, not just *how*. Its central formalization: interestingness I(T|P) = 100·V(T|P)/L(T|P) (Sec. 3.1, Eq. 6), where V is conditional proof cost in Lean 4 lines and L is premise-relative description length (Eq. 5). Claims: (1) a 27B GRPO model predicts conditional proof difficulty better than GPT-5.5 and Claude Opus 4.6 (Sec. 2, Fig. 1); (2) I correlates with extrinsic utility U0 at Spearman ρ = 0.756 after excluding zero-user theorems (Sec. 3.2, Fig. 3); (3) GRPO training on interestingness raises verified mean I from 1.76 to 7.58 across all 8 areas (Sec. 3.3, Fig. 4) while LLM-judged substantial/full mathlib containment falls 91.9% → 30.6% (Fig. 5); (4) inference-time promotion by interestingness beats no-pruning/random/proof-length rules in blinded judging — 60% ranked-first (Sec. 3.4, Tables 1–3). The Library-of-Babel framing (Sec. 1) is apt: the target is a value function for open-ended discovery.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
How does the paper define whether a math theorem is 'interesting'?
It defines interestingness as a ratio: how hard the theorem is to prove (measured as lines of Lean 4 proof code needed given available premises) divided by how long it takes to state (characters plus the definitions needed to state it). Short statements with long proofs score high — e.g., special cases of Fermat's Last Theorem rank at the top, while trivial identities like 1^n = 1 rank at the bottom.
What does the trained model actually predict?
A 27B model (Qwen3.6 fine-tuned with GRPO) predicts how many lines of Lean 4 proof code a theorem will require given a set of available premises — without writing the proof. It is more accurate than GPT-5.5 and Claude Opus 4.6 on held-out mathlib data. This difficulty estimate is the building block for the interestingness metric.
Are the generated theorems new, or copies of existing mathlib results?
The paper checks this with a blinded LLM judge on a 5-level containment rubric. Before training, 91.9% of the base model's verified statements were judged substantially or fully contained in mathlib; after interestingness training, only 30.6% are. The remaining 69.4% are verified-correct Lean theorems not present in mathlib, though judged quality in the examples appendix is competent rather than research-level.
Can this system autonomously discover important new mathematics?
Not yet. The system generates and verifies novel, non-trivial Lean theorems and can iterate on its own outputs, but the discovered statements are elementary-to-intermediate, and the authors themselves state it is unclear whether the new theorems are useful for proving future results. It is a proof of concept for the selection metric, not a discovery engine for deep mathematics.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Text extraction was lossy: Figures 1-3 render as jumbled axis labels, so exact difficulty-model numbers (e.g., per-model MAE; Spearman values read as 0.912/0.776/0.815) could not be reliably attributed to specific models.
- A middle section (part of Section 3.4 including the Figure 7 caption) was omitted in extraction; the 6-vs-10-round inconsistency between text and Appendix D table captions could not be resolved from the provided text.
- No code, checkpoint, or data availability statement was found anywhere in the extracted text; this judgment reflects the visible text only.
- Judge rubrics and prompts were fully extracted and taken at face value; judge output distributions could not be audited.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.