RAPID: Robot Agentic Programming from Demonstrations
Yuyao Liu, Jiayuan Mao, David Hsu, David Hsu, Leslie Pack Kaelbling, Tomás Lozano-Pérez
cs.RO · cs.AI · cs.CV
Quick verdict
RAPID infers task spec, primitives, and sim verifier from one demo; 75.9% vs 14.6% on 8 tasks, but thin stats and proprietary models limit certainty.
Summary
RAPID is a framework that closes the agentic coding loop for robot manipulation from a single visual human demonstration plus a language instruction. Rather than assuming ground-truth rewards, hand-engineered primitives, or resettable environments, it infers all three: it segments the demonstration into phases, infers per-phase goals and success predicates, reconstructs an interactive MuJoCo environment via SAM 3, SAM 3D, and FoundationPose real-to-sim, and generates agent-filtered scene variants for verification. Its object-centric relational representation encodes primitives as local trajectory-optimization programs (solved with CEM) achieving object-level effects, composed by relational constraints evaluated at run time; deployment needs only lightweight semantic binding. On eight contact-rich nonprehensile tasks (50 novel scenes each), RAPID reaches 75.9% average success versus 14.6% for the strongest baseline, CaP-Agent0; removing scene variants drops this to 53.2%. On LIBERO-Pro it ranks best in five of six settings against VLA and CaP methods (one tie), trailing ASPIRE only on libero-goal position perturbation (0.67 vs 0.81), and all eight tasks transfer to a real Franka arm (10 scenes each, roughly 0.4-0.9 per task). Main limitations: statistics rest on three trials with no significance testing and high variance in some cells; the pipeline depends on proprietary models (Codex GPT-5.6 Sol, SAM 3/3D, FoundationPose) with no stated code release; success criteria and prompts are undisclosed; and construction cost is only coarsely reported.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A single visual human demonstration plus a language instruction suffices to bootstrap all three ingredients of the agentic coding loop — task specification, manipulation primitives, and an interactive verification environment (Sec. I, III, V).
- The object-centric relational program representation — primitives as CEM-solved local trajectory-optimization programs composed via runtime relational constraints — is the main driver of generalization across pose, shape, material, and scene changes (Sec. IV, Table II).
- Agent-filtered scene-variant synthesis materially improves robustness, raising average success from 53.2% to 75.9% on the eight nonprehensile tasks (Table II ablation).
- Using only one debug scene and no predefined primitives, RAPID outperforms VLA and CaP baselines on five of six LIBERO-Pro settings (Table III).
- Sim-optimized contact-rich nonprehensile behaviors transfer directly to a real Franka arm on all eight tasks without per-task re-engineering (Sec. VI-D, Fig. 5).
Strengths
Addresses a genuine, clearly stated gap
Sec. I-II argue existing agentic systems (CaP-X, ASPIRE, Voyager) assume ground-truth success signals, engineered primitives, or resettable environments; RAPID derives all three from (D, l) via Algorithm 1 (Reconstruct, InferObjective, InferEffect, ProposePrimitive, VerifyRevise), making 'demonstration as the programming interface' concrete rather than rhetorical.
Large gains with component-isolated ablations (Table II)
The 6-way baseline matrix isolates the agent loop, the object-centric relational representation (OReP), and scene variants (SV): CaP 1.3%, CaP+OReP 1.1%, CaP-Agent0 14.6%, CaP-Agent0+SV 12.6%, RAPID w/o SV 53.2%, RAPID 75.9%. Notably CaP-Agent0 receives a more favorable setting (ground-truth success signals, test-time refinement) yet still loses by >60 points; SV helps only when combined with OReP.
Generality beyond the design domain (Table III, Sec. VI-D)
Honest handling of the one losing result
Stated reuse economics
Weaknesses
Statistical support is thin (Table II)
Proprietary, multi-model pipeline with no stated release
Evaluation opacity
Baseline scope confined to the LLM-coding-agent family
Real-world evidence is figure-bound and failure analysis is absent
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
RAPID: Robot Agentic Programming from Demonstrations — Preliminary Review
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is RAPID and what problem does it solve?
RAPID (Robot Agentic Programming from Demonstrations) turns a single visual human demonstration plus a language instruction into a reusable, generalizable robot manipulation program. A coding agent generates, executes, verifies, and iteratively refines manipulation code, with the task specification, the manipulation primitives, and a simulation-based verification environment all derived from the demonstration itself — no hand-engineered rewards, primitive libraries, or resettable environments are required.
How well does RAPID perform compared to baselines?
In simulation on eight contact-rich nonprehensile manipulation tasks (50 novel scenes each), RAPID averages 75.9% success versus 14.6% for the strongest baseline (CaP-Agent0). On the LIBERO-Pro benchmark it is best in five of six settings against VLA models and code-as-policy methods, and all eight nonprehensile tasks were deployed on a real Franka robot arm (10 test scenes per task, roughly 0.4-0.9 success).
Does RAPID need a reward function or human-designed primitives?
No — that is its core claim. Success predicates for each phase and the overall task are inferred by the coding agent from the demonstration and its language description, and primitives (push, flip, pivot, topple, press-move) are written from scratch as trajectory-optimization programs. Final test evaluation, however, uses a private success criterion held out by the authors that the agent never sees.
What are RAPID's main limitations?
Results rest on three trials without significance testing and show high variance on some tasks; the pipeline depends on proprietary models (Codex GPT-5.6 Sol, SAM 3, SAM 3D, FoundationPose) with no stated code or prompt release; the hidden success criteria are undisclosed; program construction takes 'tens of minutes' per task; and baselines are limited to LLM coding-agent style methods rather than the broader nonprehensile manipulation literature.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are text-only in the extraction: Fig. 5's real-world per-task values (≈0.9/0.8/0.7/0.6/0.7/0.4/0.7/0.6 for RAPID; ≤0.3 for CaP-Agent0) are read from bar labels and may be imprecise; Fig. 4 task photos and Fig. 3 program details could not be visually verified.
- No appendix is present in the extracted text, so prompts, CEM parameters, controller gains, and variant-sampling distributions cannot be assessed.
- Several cited works are 2026 preprints/blogs (CaP-X, ASPIRE, SAM 3, SAM 3D, RoboRSI) that could not be verified from the text; their characterization is taken on trust.
- The 'five of six' LIBERO-Pro claim counts one tie (0.95 vs ASPIRE 0.95 on libero-object task) as a win; strict wins are four.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.