GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning
Ruiyang Wang, Hao-Lun Hsu, Swarajh Mehta, Jiwoo Kim, Zhihao Dou, Miroslav Pajic
cs.RO · cs.AI
Quick verdict
GAVEL: graph world model lifts Qwen3-8B task success to 91.8–92.6% on BEHAVIOR-1K; well-ablated, but significance testing and code release are missing.
Summary
GAVEL is a framework that wraps LLM long-horizon robot planning in an explicit graph world model: a typed scene graph storing object relations, preconditions and effects of nine primitives, and probabilistic beliefs over unobserved object locations. Before execution, the graph rolls each LLM-generated plan forward, checks preconditions, goals, and safety, and directly repairs failures whose corrections follow from the action model (e.g., inserting NAVIGATETO or OPEN), reserving LLM re-queries for semantic failures. For multi-task instructions, retained distributions over object locations are converted into expected search costs, and the remaining task order is re-optimized after every completed task. On BEHAVIOR-1K, evaluated at the symbolic level in a lightweight 2-D executor (validated against OmniGibson on all 100 single tasks), GAVEL raises Qwen3-8B single-task success from 41.2% to 91.8% and 500-instruction multi-task success from 19.9% to 92.6%, using fewer LLM calls than feedback-only replanning (1.5 vs 2.7). Ablations attribute the gains mainly to verification plus repair; the belief-ordering contribution is efficiency-only (~4.4 m, ~5.4% distance reduction). With frontier models (GPT-5.6 Sol, Claude Sonnet 5), GAVEL reaches 99.2–99.4% on a 100-instruction subset. Limitations: symbolic-only evaluation, no significance testing, baselines reimplemented in-pipeline, and no stated code release; residual failures cluster in instruction grounding.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 3
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- An explicit graph world model can verify and repair most LLM plan failures locally, reserving LLM re-queries for semantic failures, raising Qwen3-8B single-task success from 41.2% to 91.8% on BEHAVIOR-1K (Table I).
- Graph-derived repair alone (gavel-basic) matches or beats feedback-only replanning at lower cost: 67.7% vs 55.4% for Qwen3-4B with 1 vs 3.4 LLM calls (Sec. V-E).
- Retaining full object-location distributions and reordering tasks online reduces travel distance by ~4.4 m (~5.4%, 82.45 → 78.01 m vs gavel-map) on 500 multi-task instructions without changing success (Table II).
- The world-model harness is complementary to scale: Qwen3-4B+GAVEL (75.2%) outperforms GPT-5.6 Sol (24.6%) and Claude Sonnet 5 (38.6%) alone on the 100-instruction subset, while hosted models with GAVEL reach 99.2–99.4% (Table III).
Strengths
Exemplary ablation grid isolating each mechanism
Sec. V-C defines llm-only, sayplan (feedback-only), epog (graph-edit-only), gavel-basic (repair-only, no re-query), gavel-map/static/gavel; Tables I–II cleanly separate feedback, repair, belief-collapsing and reordering effects (e.g., Qwen3-4B: 55.4% → 67.7% → 88.8%).
Benchmark scale with pre-verified specifications
Sec. V-B: 100 single tasks (avg 12.2 primitives, 240 goal predicates) and 500 multi-task instructions (avg 18.8 primitives), every reference plan verified before measuring — an order of magnitude beyond EPoG (5 tasks/46 scenes) and SayPlan (90 instructions).
Honest empirical motivation for the belief machinery
Simulator fidelity check and scaling-objection test
Efficiency argument is quantified
Weaknesses
No statistical testing anywhere
Residual-failure accounting does not add up
Baselines are in-pipeline reimplementations
Multi-task headline rests on the simplified executor
No artifacts released; repair catalogue unspecified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
GAVEL (arXiv:2609.19315, cs.RO/cs.AI) is a harness for LLM long-horizon robot planning built on an explicit graph world model: a typed scene graph (Eq. 1) storing object relations, preconditions/effects of nine BEHAVIOR-1K primitives (Eq. 2), and probabilistic beliefs over the rooms of unobserved objects (Eq. 3). It (i) rolls LLM plans forward before execution, (ii) verifies preconditions, goal completion, and a trace-level safety predicate, (iii) repairs failures whose corrections follow from the action model (Alg. 2), and (iv) reorders remaining subtasks online using expected search costs (Sec. III-B, Eq. 7). Central claims: large success gains for compact LLMs (Qwen3-8B: 41.2→91.8% single-task, 19.9→92.6% multi-task), repair beating feedback-only replanning at lower LLM cost, and complementarity with frontier models (99.2–99.4% on a 100-instruction subset). Claims are explicitly scoped to symbolic-level execution, and the conclusion honestly defers geometric feasibility to future work.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is GAVEL and what problem does it solve?
GAVEL is a framework that wraps LLM-generated robot task plans in an explicit graph world model — a scene graph holding object relations, action preconditions/effects, and probabilistic beliefs about where unseen objects are. Before execution it simulates the plan, automatically fixes mechanical errors (e.g., a missing navigation step), and only asks the LLM to replan for errors requiring semantic reasoning. On the BEHAVIOR-1K benchmark it raises Qwen3-8B success from 41.2% to 91.8% on 100 single tasks and from 19.9% to 92.6% on 500 multi-task instructions.
Does GAVEL work with frontier models like GPT or Claude, or only small local models?
Both, and the gains stack. On a 100-instruction subset, GPT-5.6 Sol goes from 24.6% success alone to 99.2% with GAVEL, and Claude Sonnet 5 from 38.6% to 99.4%, with almost no added planning time (16.5 s unchanged; 18.0 → 18.2 s). A compact Qwen3-4B with GAVEL (75.2%) outperforms either hosted model used alone, showing the harness and model scale are complementary.
Was GAVEL tested on a real robot or in full physics simulation?
Neither at scale. It was evaluated at the symbolic action level in a lightweight 2-D executor built on BEHAVIOR-1K floor plans with geometric A* navigation and frontier-based search. The 100 single-task outcomes matched OmniGibson exactly, but the 500 multi-task results come from the simplified executor only, and low-level motion feasibility is not modeled — the authors flag this as a limitation.
Is code or data available for GAVEL?
The paper does not state any code, prompt, or model-weight release. It describes training a location-prediction network (RSN) on 11,218 scene placements and two LoRA adapters on 8,000 synthetic instruction pairs, but provides no repository link, so full reproduction currently requires rebuilding the executor, prompts, and learned components from the paper's descriptions.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–3 (pipeline overview, failure breakdown) are referenced but not analyzable from extracted text; Fig. 3's per-step failure counts are unavailable.
- Supplementary material (video, prompts, repair edit catalogue) was not part of the extracted text.
- Tables were rendered as inline text; minor value-transcription ambiguity is possible though internally consistent.
- Review covers arXiv v1 (2609.19315); peer-review status unknown.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.