MAPLE: Memory-Augmented Planning with Language and Evolution
Kesheng Chen, Yamin Hu, Wenjian Luo
cs.AI
Quick verdict
Persistent-state LLM agent for optimization under NL updates; thorough ablations, honest stats, but advantage shown mainly on its own synthetic benchmark.
Summary
The paper introduces 'live dynamic optimization': maintaining an executable optimization program through successive natural-language revisions while preserving accepted decisions and reusing search state. MAPLE retains an LLM-authored Workbench (build_problem/evaluate), accepted plans, event history, and candidate populations. Typed Search-Space Scaffolding binds decision types to fixed genetic operators (GA for scalar, NSGA-II for Pareto tasks); Live Problem Decomposition localizes edits; Live State Memory resolves references to earlier updates and accepted plans; a semantic selector chooses between warm restarts from repaired history and full reinitialization. The accompanying NLDO benchmark has 15 trajectories, 195 states, and 180 updates spanning selection, scheduling, rostering, routing, and cloud placement. MAPLE completes all trajectories with online scalar quality 0.951 and Pareto hypervolume 0.875, far above adapted baselines (best: Persistent ReAct at 0.501/0.042). Ablations show typed scaffolding is critical (removal cuts feasibility from 100% to 80.6% and HV to 0.665); the restart selector's edge over Always Warm is small (0.010 HV, sign test p=0.219), and a simple table-change rule reproduces its choices. Main limitations: all baselines are author-adapted, the benchmark is synthetic and self-designed, MAPLE trails OptiMUS on static tasks (59.3% vs 69.5%), and the primary model checkpoint is no longer available.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Maintaining executable optimization state (program, accepted plans, events, candidate populations) across natural-language updates yields far higher trajectory completion and plan quality than adapted isolated-request or stateless baselines on NLDO (100% solve rate, 0.951 DS quality, 0.875 DM HV vs best baseline 0.501/0.042).
- Typed Search-Space Scaffolding is the primary driver of update validity: removing it cuts feasibility from 100.0% to 80.6% and mean HV from 0.879 to 0.665 (Table 3).
- Warm reuse of repaired candidates helps under benign revisions while full reinitialization helps after disruptive replacements, and a gated semantic selector captures this trade-off (edge over Always Warm: 0.010 HV, p=0.219).
- Some historical-binding requests require both stored event records and the accepted plan; neither source alone resolves them (Table 20).
Strengths
Novel, practical problem formulation
Formalizes live dynamic optimization (state St = (Dt, Wt, At, Pt) under cumulative NL updates, Section 2), cleanly distinct from one-shot NL-to-model systems (OptiMUS, ORLM) and classical dynamic evolutionary optimization (Branke 2002); NLDO operationalizes it with separate continuity (strict prefix) and saved-plan-quality metrics.
The typed-scaffolding ablation is convincing
Removing TSS drops feasibility from 100.0% to 80.6% and mean HV from 0.879±0.014 to 0.665±0.004 over six DM episodes, ten seeds (Table 3); the 140 invalid outputs trace to identifiable missing checks (vehicle uniqueness, GPU feasibility flags; Table 21).
Unusually honest statistical and protocol reporting
Well-designed historical-grounding tests
Conservative state-commitment design
Weaknesses
All NLDO baselines are author-adapted reimplementations
Protocol strictness conflates compliance with solution quality
Restart-selector evidence is thin relative to its billing
Advantage confined to a self-built synthetic benchmark
Backbone sensitivity and checkpoint unavailability
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
MAPLE addresses "live dynamic optimization": keeping an executable optimization program, accepted plans, event history, and candidate populations coherent across a stream of natural-language revision requests. The system couples an LLM-authored two-function Workbench (build_problem, evaluate) with fixed machinery: Typed Search-Space Scaffolding (TSS) binds declared decision types (binary, permutation, assignment, real, integer) to fixed GA/NSGA-II operators or an LP/MILP interface; Live Problem Decomposition (LPD) localizes each update to table edits or one of the two functions; Live State Memory (LSM) resolves references to earlier events and accepted plans and enforces the retrieved assignments during search; a semantic restart selector picks Warm (at most 50% repaired history) or Full (fresh population), subject to a fixed evidence gate (Algorithm 1). The authors claim three things: persistent executable state beats isolated-request agents on continuity and quality; TSS is the main validity driver; and reuse decisions can be made sensibly across benign and disruptive revisions. Evidence comes from the new NLDO benchmark (15 trajectories, 195 states, 180 updates across selection, precedence scheduling, rostering, routing, and cloud placement) plus static tasks NLP4LP-hard and BWOR20.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is MAPLE?
MAPLE is an LLM-based agent that maintains an optimization problem and its solutions across successive natural-language change requests. It keeps an executable program (two LLM-authored functions), accepted plans, an event history, and candidate populations, and uses fixed genetic algorithms (GA/NSGA-II) or an LP/MILP interface to re-solve after each accepted update.
What is the NLDO benchmark?
NLDO is a synthetic benchmark introduced with the paper: 15 trajectories of 13 states each (t00-t12), 195 states total, and 180 natural-language updates across five domains - selection/allocation, precedence scheduling, rostering, routing, and cloud-resource placement - with single- and multi-objective views and held-out reference solutions.
How well does MAPLE perform compared to other methods?
On NLDO it completes all 15 trajectories with a 100% solve rate, online scalar quality 0.951 and Pareto hypervolume 0.875, while the best adapted baseline (Persistent ReAct) reaches 0.501 and 0.042. Note that all baselines were reimplemented by the authors and part of the gap reflects strict output-protocol compliance rather than pure solution quality.
Is the code or benchmark available?
The paper lists two GitHub repositories and a live demo, and describes a released evaluation package containing tasks, prompts, references, and seed-level records. However, the primary language model (DeepSeek-V4-Pro Preview) is no longer available, so exact replication of the headline numbers is not possible; a Kimi k2.7 replication is reported.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with [...sampling gap...] blocks (covering part of the Section 4 narrative, the tail of Algorithm 2 into A.5, parts of A.11/A.13, and context around Table 34) and ends mid-sentence in Appendix A.18; all numbers cited in this review come from content present in the excerpt.
- Per-episode tables for the scalar domains (P001-P009) are not visible in the excerpt, so scalar claims rely on aggregate values (e.g., 0.951 DS online quality).
- The linked repositories, live demo, and evaluation package were not inspected; code/data availability is reported as stated in the paper.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.