World Action Agent: Harnessing VLMs for Robot Manipulation via World Action Rehearsal
Yehang Zhang, Haojian Huang, Yifan Chang, Jianchong Su, Bohan Zhou, Yingjie Xu, Wosong Chen, Tianhao Zhou, et al.
cs.RO · cs.AI
Quick verdict
WAA reports SOTA 75.6% on LIBERO-Pro via a VLM harness with contact views, action rehearsal, and in-view correction; strong system, but no statistical testing.
Summary
The paper introduces World Action Agent (WAA), a multi-agent harness through which general-purpose vision-language models (VLMs) pilot robots with basic primitives inside a visual action workspace. Three mechanisms define the workspace: Contact views selected automatically from scene geometry to center observation on the interaction; action rehearsal, where every action is an editable proposal previewed by an Imagination Agent and checked by a motion planner (cuRobo) before execution; and in-view correction, which converts image-space drags into bounded end-effector displacements using view calibration. WAA evolves a multimodal skill library from LIBERO-90 expert videos and human teaching through a Learner-Editor-Reviewer pipeline driven by GPT-5.5, and distills its interaction traces into a Qwen3.5-9B pilot. With Gemini 3.7 Flash as backbone, WAA reaches 75.6% average success on LIBERO-Pro (against ASPIRE at 72.0%), frozen skills transfer to robosuite with 100% on all three tasks, and fine-tuning raises the 9B pilot's out-of-domain success from 1.7% to 43.3%. Main limitations: no statistical tests or error bars; most baselines are quoted from other papers under different protocols (rerun Show-Harness used 30 episodes per split versus 60 for WAA); dependence on proprietary models with no code release stated; and missing ablations isolating the three workspace properties.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A visual action workspace combining interaction-centered Contact views, action rehearsal, and in-view correction lets general-purpose VLMs pilot robots with basic primitives.
- WAA with skills evolved only from LIBERO-90 achieves state-of-the-art 75.6% average success on LIBERO-Pro, outperforming VLAs, code-as-policy agents, and a same-backbone visual harness.
- The frozen skill library transfers to robosuite without further learning, lifting cube restacking from 60.0% to 100.0%.
- Fine-tuning Qwen3.5-9B on harness interaction traces raises its out-of-domain success from 1.7% to 43.3% while sub-agents stay on the proprietary backbone.
- Evidence-based Learner-Editor-Reviewer review with visual references prevents skill-library degradation and enables one-demonstration skill acquisition (10/10 on the stove task).
Strengths
Coherent closed-loop workspace design
The three properties share one calibrated Canvas (Sec. 3.2): Contact views chosen by Eq. (2) enforce orthogonal horizontal projections so offsets are readable along two independent axes; rehearsal couples VLM judgment with cuRobo feasibility checks; drags map to bounded TCP deltas (0.03/0.08 m limits, Table 4). The full tool API (Tables 4-6) is documented precisely enough to reimplement the interface.
Evidence-gated skill evolution
The Learner reads source frames but not existing skill text; the Editor must cite frames; the Reviewer independently checks edits. Appendix A.3 shows this concretely: the Reviewer rejects treating knob motion as success, and the accepted stove skill yields 10/10 on the LIBERO-Pro goal-swap suite from a single demonstration.
Strong, multi-axis empirical results
Transparent cost and failure reporting
Weaknesses
No statistical treatment of headline margins
Mixed evaluation protocols across baselines
Missing component ablations
Skill-source/evaluation proximity undercuts generalization framing
Reproducibility depends on closed models; no release stated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
WAA (Secs. 1, 3) is a systems/method paper for simulation-based robot manipulation: general-purpose VLMs (Gemini 3.7 Flash) pilot a robot arm through basic tools inside a visual action workspace, rather than being fine-tuned into a VLA or emitting code. Four claims are tested: (1) the workspace's three properties — interaction-centered Contact views, action rehearsal, and in-view correction — suffice for VLMs to act as robot pilots; (2) skills evolved only from LIBERO-90 yield state-of-the-art 75.6% average success on LIBERO-Pro (Table 1); (3) the frozen library transfers to robosuite (Table 3); (4) interaction traces teach a 9B VLM to pilot the same harness (Fig. 5). No real-robot experiments appear in the provided text.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is World Action Agent (WAA)?
WAA is a multi-agent framework that lets general-purpose vision-language models (VLMs) such as Gemini drive a robot arm using basic tools. All decisions happen in a visual workspace with interaction-centered camera views, editable action proposals that are rehearsed against a motion planner before execution, and drag-based corrections directly in the camera view. It also builds a library of multimodal skills from expert videos and human teaching, and can fine-tune smaller VLMs from its own interaction logs.
What success rate does WAA achieve on LIBERO-Pro?
With skills evolved only from LIBERO-90 demonstrations, WAA (Gemini 3.7 Flash backbone) reports 75.6% average success across six LIBERO-Pro splits, ahead of ASPIRE (72.0%), RATs (43.8%), and end-to-end VLAs such as OpenVLA (0.0%) and pi0.5 (12.8%). Without skills it reaches 28.9%, and the same frozen skills transfer to robosuite tasks at up to 100% success.
Does WAA work on real robots?
The reported experiments are in simulation (LIBERO-Pro and robosuite). The system is designed so scene point clouds can come from RGB-D cameras or VGGT reconstruction rather than the simulator, and RGB-D inputs only lower LIBERO-Pro success from 75.6% to 71.1%, but the paper text reports no real-robot hardware experiments.
Can a smaller, open VLM run the WAA harness?
Partially. The authors fine-tuned Qwen3.5-9B on 1,774 tool-call decisions from 112 successful Gemini-driven episodes; the 9B model replaced only the main agent while sub-agents stayed on Gemini. This raised out-of-domain success from 1.7% to 43.3% and in-domain success from 0.0% to 55.0% on LIBERO-Pro.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-10 are available only as captions in the extracted text; interface visualizations and qualitative claims could not be verified.
- Baseline numbers for ASPIRE, RATs, CaP-Agent0, OpenVLA, and pi0.5 are quoted from their papers; the original evaluation protocols could not be cross-checked here.
- Some quantitative details (per-task breakdowns, Eq. 2 hyperparameters) do not appear in the provided text and may or may not exist in the full PDF.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.