Evolutionary Ensemble Search: Council-Guided Program Evolution with Persistent Memory
Juan P. Madrigal-Cianci, Eshan Chordia
cs.NE · cs.AI
Quick verdict
EES pairs role-specialized LLM councils with evolutionary ML-program search; a candid MLE-bench Lite development campaign reports 19/22 medal artifacts, but with no ablations, baselines, or engine code.
Summary
The paper presents Evolutionary Ensemble Search (EES), a system in which a role-specialized council of LLM experts (discovery, methodology, memory, modeling, code review) converts task evidence into structured directives that an orchestrator routes to specialists and an evolutionary engine. The engine selects measured parents, applies diagnostic code mutation, typed pipeline edits, and crossover, requires each child to acquire its own fitness by execution under a shared comparison protocol, retains alternatives in archives, and gates ensembling on validation improvement. Experience is stored at three timescales: operator credit, session memory, and a task-indexed cross-run lesson library. Three execution profiles (multi-island, nested, benchmark) share these interfaces. The evidence is a versioned MLE-bench Lite development ledger: medal-threshold artifacts on 19 of 22 tasks (11 gold, 5 silver, 3 bronze), spanning text, image, tabular, audio, and scientific-geometry procedures. The authors are explicit that grades were visible between runs, four tasks used external sources (Pizza reaches 1.00000 via target lookup), confirmations mix replays of identical submission hashes with new runs, and the engine is proprietary. No ablations, matched baselines, or consolidated compute accounting are provided, so the marginal contribution of council review, operator credit, and memory mechanisms is not established; the paper reads as an unusually well-hedged system report and development case study rather than a controlled evaluation.
Review complete
Assessment at a glance
6
risk flags identified
● high 3
● medium 3
● low 0
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- EES couples a role-specialized LLM council to an evolutionary engine that mutates measured parent programs, executes children under a shared comparison protocol, and gates ensembling on validation improvement.
- Experience adapts the search at three timescales (operator credit, session memory, task-indexed lesson retrieval) without updating language-model weights.
- A public, hash-pinned MLE-bench Lite development ledger records medal-threshold artifacts on 19 of 22 tasks (11 gold, 5 silver, 3 bronze) across six procedural modalities.
- The aggregate is explicitly an achieved development result, including grade feedback between runs and some external-source routes, and does not estimate blind autonomous performance or per-component contribution.
Strengths
Explicit measurement contracts and candor about result scope
Section 3 defines a comparison protocol and treats timeouts/exceptions as typed failures rather than low fitness; Section 5.1 explicitly reframes the headline 86.36% as an achieved development result with inter-run grade feedback, not a blind autonomous-agent success rate.
Transparent outcome accounting beyond the headline
Tables 4 and 7 separate best artifacts from confirmations, expose medal downgrades (APTOS silver best vs bronze confirmations; NOMAD gold C1 vs silver C2-C3), and report hash counts showing 57 medal confirmation records collapse to only 46 distinct task-submission hashes, with four tasks repeating a single hash across C1-C3.
Versioned, hash-pinned evidence trail
Genuinely cross-modal executable procedures
Statistically careful hedging of adaptive selection
Weaknesses
No ablations, baselines, or component attribution
Proprietary engine makes search irreproducible
Benchmark-protocol contamination of the aggregate
Weak and partially circular confirmation protocol
Formal mechanisms specified but never exercised in evidence
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Evolutionary Ensemble Search (EES)?
EES is a system for automating machine-learning engineering. A council of role-specialized LLM experts (discovery, methodology, memory, modeling, code review) reviews task evidence and emits structured search directions; an orchestrator routes them to specialists and an evolutionary engine. The engine mutates and crosses over previously measured programs, and every child program must be executed to earn its own validation score before it can be selected, archived, or ensembled.
What score did EES achieve on MLE-bench Lite?
The public development ledger records medal-threshold artifacts on 19 of 22 MLE-bench Lite tasks (86.36%), with best outcomes of 11 gold, 5 silver, and 3 bronze. This is a development result: competition grades were visible between runs and influenced later engineering, four tasks used external sources (one scoring full marks via target lookup), so it is not a blind autonomous-agent success rate.
Is the EES code available?
No. The paper states the EES engine is proprietary and not included. A pinned public repository provides the results ledger, solution notes, selected artifacts, and reproduction instructions, so outsiders can check the ledger and regrade submitted prediction files, but cannot reproduce the search itself.
Did the system use external data or cheat?
Four tasks explicitly used external sources or historical lookup: Pizza (external target lookup, scoring 1.00000), Dog Breed (external image lookup), Spooky Author (a Gutenberg corpus), and Right Whale (historical lookup or exploit discovery). The paper flags these, notes that retrieving withheld targets is not evidence of predictive generalization, and says an unmodified grader does not certify protocol compliance for any task.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Review is based solely on the embedded extracted text; Figure 1 is available only as its caption-like text and could not be visually assessed.
- Tables were reconstructed from PDF text extraction; cell alignment was verified against the surrounding prose but minor formatting nuances may be lost.
- External claims (repository commit, blob/SHA-256 identifiers, ledger contents) were taken on trust; no network access was used to verify them.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.