LLM-Guided Program Evolution for Circle Packing: Breaking 10 Packomania Records for $28
cs.AI
Quick verdict
Claims a $28 LLM evolution loop broke 10 packing records, but Table 1 shows its classical seed solver broke all 10 at iter 0; LLM gains were only ~1% overall.
Summary
The paper presents Discovery Loop, a roughly 400-line Python system in which an LLM (Claude Fable 5.1) iteratively rewrites a complete optimization solver for the Packomania csqv circle-packing problem, guided by a scoreboard of results, a history of the last 12 ideas, and an independent zero-tolerance feasibility verifier. Running 15 iterations overnight on a consumer PC for $27.72 of LLM cost, it reports new best-known solutions for 10 of 12 targets (N = 101, 102, 103, 105–109, 111, 114), improving prior records by 2.4%–5.4%, with the results accepted by the Packomania maintainers. Crucially, Table 1 attributes every record to iteration 0 — the classical seed solver (multi-start penalty L-BFGS-B with LP-optimal radii) — before any LLM proposal; the LLM loop raised the 12-target total only from 59.39 to 59.98 (about 1%). The table is transparent about this, but the title and abstract frame the records as products of LLM-guided evolution. Strengths include per-iteration cost accounting ($4.96 productive phase versus $22.76 diminishing phase, a 130× jump in cost per unit improvement), open-source code and submission records, and documented algorithmic ideas such as hexagonal-lattice initialization and KKT-Newton polishing. Main limitations: no equal-budget baseline (e.g., longer seed-only multi-start), a single run with no variance estimates, an in-sample backtest of the plateau-stopping rule, and an MIPLIB experiment reported only as a 75% code-generation failure rate.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 2
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 16 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Discovery Loop reproduces the core AlphaEvolve evolutionary loop in ~400 lines with a single LLM, a single consumer PC, and $27.72 of LLM cost.
- The system produced 10 new best-known csqv packings (N = 101, 102, 103, 105–109, 111, 114), improving prior records by 2.4%–5.4%, accepted by Packomania.
- Per Table 1, all 10 records were first achieved at iteration 0 by the classical seed solver, with the LLM loop adding only ~1% to the 12-target total (59.39→59.98).
- An adaptive plateau detector (W = 4, θ = 0.01) would have cut cost by ~50% while sacrificing ~0.01% of solution quality, based on backtesting the actual run.
- The LLM generated genuine algorithmic innovations: basin hopping with contact polish, hexagonal-lattice initialization, island models, KKT-Newton polishing, and defect migration (§4.5).
Strengths
Transparent attribution data in Table 1
The 'First Improved' column explicitly credits all 10 records to 'Iter 0 (seed)', letting readers see that the classical seed solver — not the LLM loop — broke the records; the internal arithmetic is consistent (Table 2 costs sum to $27.72; $4.96 + $22.76 phase split; 130× cost-per-improvement ratio in §4.4).
Independent verification chain
§3.4 specifies a verifier that shares no code with the solver, applies zero tolerance plus a strict feasibility shrink, and results were accepted by Packomania (Abstract, §4.2) — the right standard for a benchmark where margins live in the fourth decimal.
Genuine minimalism and openness
Documented algorithmic content of LLM proposals
Honest limitations section
Weaknesses
Title/abstract attribution conflicts with the paper's own Table 1
No equal-budget control for the LLM's marginal value
Single run, no variance, in-sample plateau backtest
MIPLIB generalization evidence is underreported
Unexplained target selection and prior-record context
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Wes Sander (Practical Systems) presents Discovery Loop, a ~400-line Python reimplementation of the AlphaEvolve paradigm: each iteration, one LLM (Claude Fable 5.1, accessed via Claude CLI) proposes a complete replacement solver, guided by a scoreboard, a 12-idea history, and an independent zero-tolerance verifier (§3.2). The headline claims are: (1) 10 new best-known solutions on the Packomania csqv circle-packing benchmark (N = 101, 102, 103, 105–109, 111, 114), with gains of 2.4%–5.4% accepted by Packomania; (2) total LLM cost of $27.72 across 15 iterations overnight on an i7-13700KF consumer PC (§1, §4.1); (3) a plateau detector (W = 4, θ = 0.01) that would halve cost at ~0.01% quality loss (§3.5); (4) evidence that the AlphaEvolve paradigm is accessible to individuals (§1.1, §6.3).
The paper's most important interpretive fact sits in Table 1's 'First Improved' column: every one of the 10 records is credited to iteration 0 — the classical seed solver (multi-start penalty L-BFGS-B with LP-optimal radii), before any LLM proposal. Table 2 shows the LLM loop raising the 12-target total from 59.39 to 59.98 over iterations 1–14, i.e., about 1%.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Did this paper really break 10 circle-packing records for $28?
It reports new best-known solutions for 10 large-N instances of the Packomania csqv benchmark (N in 101–114), improving prior records by 2.4%–5.4% at $27.72 of LLM cost, with acceptance by Packomania. However, its own Table 1 shows the classical seed solver broke all 10 records before the LLM proposed anything; the LLM loop added only about 1% on top of the 12-target total.
What is the Packomania csqv problem?
Packomania is a long-maintained database (since 1998) of best-known packings of N circles in a unit square. The csqv variant lets each circle's radius vary and scores by the sum of all radii, which is maximized subject to all circles fitting inside the square without overlapping.
Can I reproduce the Discovery Loop results?
The author states all code, solvers, results, and submission records are open source on GitHub, and record values can be checked against Packomania itself. Verification is straightforward, but the exact LLM iteration trajectory depends on a proprietary model accessed via a CLI with no reported seeds or temperature settings, so runs will not repeat identically.
How does this differ from AlphaEvolve?
AlphaEvolve (Google DeepMind) uses a distributed evaluation cluster, an evolutionary program database, and multiple LLM calls per iteration. Discovery Loop uses one LLM call per iteration, a single champion with no population, and a consumer PC, trading scale for accessibility — though, per its own Table 1, most record-breaking came from its classical seed solver rather than the LLM loop.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text ends cleanly at the References (page 8) with no sign of truncation; equations (1)–(3) and table formatting are flattened by PDF extraction but all cited numbers remain legible, so no content-based caveat applies.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.