Generalizable Robotic Insertion with World Models
Nicklas Hansen, Iretiayo Akinola, Yijie Guo, Jie Xu, Bingjie Tang, Hao Su, Xiaolong Wang, Abhishek Gupta, et al.
cs.RO · cs.CV · cs.LG
Quick verdict
Sim-only world-model insertion generalist: 56% zero-shot on unseen objects vs 7% baseline, clear scaling; strong evidence, but no real-robot validation.
Summary
The paper introduces InsertionWM, a model-based reinforcement learning framework for robotic peg-in-hole insertion that generalizes across object geometries. Building on TD-MPC2, the authors train a single latent world model from raw 96×96 wrist-camera depth images and 24-D proprioception, planning actions via MPPI, without CAD models or object IDs at deployment. Using the 100 assemblies from AutoMate (last 10 held out), specialist policies reach 79.78% success versus 70.31% for a reproduced AutoMate; a generalist trained on 90 tasks reaches 55.0% in-domain versus 15.6% for AutoMate on 80 tasks, and 56% zero-shot success on 10 unseen objects versus 7% for the model-free baseline, with performance scaling monotonically from 27% at 5 training objects. Finetuning the generalist on held-out assemblies is markedly more data-efficient than training specialists from scratch and occasionally exceeds their asymptotic performance. Ablations show vision is critical, curricula help, and expert rollouts add only marginal gains. Limitations are acknowledged: entirely simulation-only (Isaac Gym), sensitivity to visual occlusion, a slight in-domain decline as tasks grow, and unresolved sim-to-real challenges such as depth noise and contact fidelity. Statistical reporting is limited to means over 100 trials per assembly, with no confidence intervals or significance tests, and the AutoMate generalist comparison uses numbers from the original paper under unknown train/test splits.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A single TD-MPC2 world model trained on up to 90 insertion tasks achieves 56% zero-shot success on 10 unseen assemblies versus 7% for the model-free AutoMate generalist.
- Zero-shot and in-domain performance scale monotonically with the number of training assemblies, from 27% at 5 objects to 56% at 90 objects.
- Model-based RL outperforms model-free PPO-based AutoMate for both specialists (79.78% vs 70.31% across 100 assemblies) and generalists (55.0% vs 15.6% in-domain).
- Finetuning the 90-task generalist on held-out assemblies is substantially more data-efficient than training specialists from scratch, occasionally with better asymptotic performance (objects 1036, 1041).
- Raw wrist-depth vision plus proprioception, without CAD models or object IDs at deployment, suffices for geometry-aware insertion of unseen parts.
Strengths
Systematic scaling study
Figure 1 evaluates five dataset sizes (5/15/45/75/90 training objects) with 100 trials per assembly; zero-shot success rises monotonically 27%→40%→50%→53%→56%, and in-domain reaches 55.0% vs AutoMate's 15.6% on 80 tasks — the paper's most convincing evidence.
Deployment-relevant problem formulation
Unlike AutoMate's generalist, which consumes ground-truth CAD point-cloud embeddings (including for test assemblies) at deployment, InsertionWM acts purely from raw wrist depth and proprioception (Sec. V), removing privileged object information from the control loop.
Fair specialist comparison with disclosed provenance
Candid limitations and useful ablations
Weaknesses
No real-robot experiments
Cross-paper generalist comparison
Thin statistical reporting
Zero-shot scope open to misreading
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
InsertionWM targets generalizable peg-in-hole insertion: one policy for many object geometries, zero-shot transfer to unseen ones. The central claims (Abstract, Sec. I) are that (1) this is the first fully data-driven system assembling unseen objects, (2) a single world model trained on up to 90 tasks achieves 56% zero-shot success versus 7% for the model-free baseline, (3) performance scales monotonically with training-set size (27%→56% over 5→90 objects), and (4) finetuning the generalist is far more data-efficient than training from scratch. Every experiment is simulation-only (Isaac Gym, Franka Panda, Secs. III-IV), and the claim's scope must be read accordingly.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is InsertionWM?
InsertionWM is a model-based reinforcement learning framework (built on TD-MPC2) that trains a single world model to perform peg-in-hole insertion across many object geometries, using only wrist-camera depth images and robot proprioception. At deployment it needs no CAD models, object IDs, or pose estimation.
Does InsertionWM work on a real robot?
No. All reported experiments are in simulation (Isaac Gym) on a Franka Panda setup. The authors explicitly list sim-to-real transfer as an open challenge, citing noisy depth on small parts, imperfect contact simulation, and real-time visual processing requirements.
How does it compare to AutoMate?
On the same 100-assembly benchmark, its specialist policies reach 79.78% success versus a reproduced AutoMate's 70.31%. Its 90-task generalist reaches 55.0% in-domain versus AutoMate's 15.6% (on 80 tasks), and 56% zero-shot on unseen objects versus 7%. Caveat: the AutoMate generalist numbers come from the original paper under unknown train/test splits.
What does 'zero-shot on unseen objects' mean in this paper?
The 10 test assemblies come from the same 100-assembly dataset as the training data and were never seen during training. The geometry is new to the model but drawn from the same distribution — it does not mean the system handles arbitrary real-world objects.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Extracted text is truncated: portions of Sections IV and VI-A and Figure 7 details are omitted; assessments rely on the surviving text.
- Figures 1-7 are not analyzable from the extraction; cited values come only from captions and body text.
- No code or data availability statement appears in the extracted text; a project page or release may exist outside the excerpt.
- Absence of real-robot experiments is inferred from the extracted text; no hardware section appears in what was provided.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.