World State Generator
Sungheon Jeong, Sanggeon Yun, Ryozo Masukawa, Haleh Alimohamadi, Mahdi Imani, Mohsen Imani
cs.AI
Quick verdict
Trains plan repair on verified failure-repair pairs from synthetic worlds; 3.4–31.9pt zero-shot gains on 7 benchmarks. Solid pipeline; parity claim unproven.
Summary
The paper trains language agents to repair plans after a step fails, using verified failure-repair pairs rather than success-only trajectories. Building on the State-Centric Decision Process (SDP) runtime, where each plan step is a natural-language predicate checked against the world, the authors construct synthetic worlds in 7 domains (physical, software, web search, tool use, constraint, reasoning, math), each admitted only if goal reachability is proven. Three open ~30B models ran 226,316 episodes; the world refuted at least one state in 75.3%, yielding 775,180 repair calls of which 238,563 (30.8%) are kept, each pairing a refutation with a rewritten chain the world later certified at least in part. WSG is a LoRA fine-tune of Gemma4-31B and Qwen3.6-27B trained by supervised learning on repair calls only; first plans remain prompted. Zero-shot on 7 public benchmarks (τ2-bench, ScienceWorld, ToolSandbox, PlanBench-XL, REAL, DeepPlanning, AgentBoard PDDL), WSG gains 3.4–31.9 points over the SDP prompting baseline, with gains concentrated in post-refutation recovery. Limitations: no significance testing or seed variance; proprietary comparisons quoted from published results under different harnesses, and the conclusion's parity claim contradicts §5.2's own gaps on PlanBench-XL and DeepPlanning; realizer and validator left untrained (45% and 20% of remaining failures); no stated code or data release.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 5
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Verified failure-repair pairs from program-enforced synthetic worlds teach plan repair that transfers zero-shot to 7 unseen public benchmarks.
- WSG raises end-to-end success by 3.4–31.9 points over prompted SDP for two ~30B open backbones (Gemma4-31B, Qwen3.6-27B).
- SFT on refutation-conditioned repair pairs avoids the first-attempt collapse SCoRe reports for self-correction SFT, because every target is tied to evidence the first attempt lacked.
- Gains concentrate in recovery after refutation, and on 6 of 7 benchmarks WSG uses fewer replans and less wall-clock time than prompting.
Strengths
Novel supervision signal with targeted filters
§4.1/Eq. 6 keeps a repair only if the rewritten chain certified at least one world state, and drops repairs restating already-refuted chains or naming actions — precisely the patterns behind SCoRe's first-attempt collapse, enabling SFT-only training (§4.2).
Programmatic grounding on both sides of each pair
Worlds are admitted only after reachability is proven by reference route or exhaustive search (Eq. 5), the goal check χg is program-executed, and refusal/repair outcomes are decided by the world rather than human judgment (§4.1, Table 1).
Consistent zero-shot transfer with mechanism localization
Candid failure attribution and cost honesty
Weaknesses
No statistical rigor
Parity claim contradicts own results
Uncontrolled proprietary baselines
'Verified' is partially LLM-judged
Missing controls and release plan
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
WSG targets one operator: REPLAN, the repair call of the State-Centric Decision Process (SDP) runtime [10], in which plan steps are natural-language predicates checked against the world and a failed check yields a refutation τ = (refuted state, observation). The paper makes three claims: (1) a pipeline manufacturing verified failure–repair pairs at scale — 226,316 episodes across 7 synthetic world families, 775,180 repair calls, 238,563 kept (30.8%); (2) that supervised fine-tuning on such pairs avoids the first-attempt collapse SCoRe [12] reports for self-correction SFT, because every target is conditioned on refutation evidence the first attempt lacked (§4.2); and (3) zero-shot transfer — +3.4 to +31.9 points of end-to-end success over prompted SDP for two ~30B open backbones on 7 public benchmarks (Table 3). The scope is deliberately narrow: first plans remain prompted, and the realizer and validator stay untrained (§6).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is World State Generator (WSG)?
WSG is a fine-tuned language model that rewrites an AI agent's plan after a step fails. Plans are written as checkable statements about the world; when the environment refutes one, WSG reads that refutation and rewrites the remaining steps. It was trained on roughly 226K episodes of program-verified failure-repair pairs collected in 7 synthetic world domains, using LoRA on Gemma4-31B and Qwen3.6-27B.
How much does WSG improve agent performance?
On 7 public agent benchmarks (τ2-bench, ScienceWorld, ToolSandbox, PlanBench-XL, REAL, DeepPlanning, AgentBoard PDDL), WSG improved end-to-end success by 3.4 to 31.9 points over the same backbone models with prompting, in zero-shot transfer. The largest gain was on PlanBench-XL with Qwen3.6-27B (16.1 to 48.0).
Why train on failures instead of successes?
Existing agent training corpora mostly keep successful trajectories, so models never learn what to do when the world refuses a step. WSG's training pairs each verified failure with the repair that the world later certified at least partly, teaching the model to use the failure's evidence rather than reword the refused step and hit the same refusal again.
Is code or data available for this paper?
The paper does not state any code, corpus, or model release. The 7 synthetic world families, the 238,563 kept repair pairs, and the trained LoRA adapters are described but their public availability is not announced, so independent reproduction currently requires rebuilding the entire pipeline.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 is available only through its caption; the pipeline diagram itself could not be visually inspected.
- No appendix or prompt templates appear in the extracted text; validator and realizer prompt designs cannot be audited.
- Table 3's proprietary column gives only names and scores from publications, so harness details of those baselines could not be verified from this text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.