Accelerating Visual Policy Learning with Sampling-Based Model Predictive Control
Yilang Liu, Haoxiang You, Qian Wang, Daniel Rakita, Ian Abraham
cs.RO · cs.AI · cs.LG
Quick verdict
SGPS pairs sampling-based MPC with first-order policy gradients for single-GPU visual policy learning and zero-shot Go2 transfer; promising, thin statistics.
Summary
The paper proposes SGPS (Sampling-Guided Policy Search), a framework that accelerates visual policy learning for legged robots by combining sampling-based model-predictive control with first-order policy gradients (FoPG) through differentiable simulation. Sampling MPC in the DIAL-MPC style first generates a task reference trajectory, from which behavior cloning initializes the policy; training then alternates recurring action-target refinement — sampled segments must pass a score-acceptance gate — with short-horizon FoPG tracking updates under perturbed resets and randomized dynamics. Building on the decoupled FoPG formulation, which excludes rendering from the computation graph, depth-based visual specialists train directly without a privileged state teacher. On one RTX 4080, SGPS trains Go2 trot/crawl/hurdle and G1 crate-push/box-carry specialists; the authors report PPO needs roughly 196 times more environment interactions on the state-based trot, and refinement lifts visual returns (Go2 trot about 16,000 vs 11,000 near 4.1M steps; G1 carrying about 26,500 vs 20,500 near 6M steps). A DAgger-distilled unified Go2 policy transfers zero-shot to hardware, autonomously switching trot/crawl/hurdle from onboard depth. Limitations: results are read from plots with three seeds and no statistical tests; G1 manipulation is heavily simplified (rail-constrained crate, soft wrist attachments) and simulation-only; each task requires hand-crafted sampling objectives and references; no code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Coupling recurring sampling-MPC action-target refinement with decoupled first-order policy gradients learns visual legged-robot policies faster than FoPG or behavior-cloning initialization alone.
- Decoupled FoPG, which excludes rendering from the computation graph, enables direct depth-based specialist training without a privileged state-policy teacher.
- On a single RTX 4080, SGPS trains Go2 trot/crawl/hurdle and G1 push/carry specialists, with PPO requiring about 196x more environment interactions on the state-based trot task.
- A DAgger-distilled unified Go2 visual policy transfers zero-shot to hardware and autonomously switches trot/crawl/hurdle behaviors using only onboard sensing and a velocity command.
Strengths
Genuinely modest compute envelope
Refinement search, FoPG rollouts, and evaluation all run on a single RTX 4080 (Sec. V-A.b); state-based trot reaches returns PPO only matches after ~196x more environment interactions (Fig. 3c), with wall-clock and env-step axes reported separately
Honest, well-controlled ablation
'SGPS, no refinement' shares reference generation, BC init, architecture, reward, and FoPG optimization with the full method (Sec. V-C.a); authors concede state-trot gains are modest and that 'much of the efficiency relative to PPO remains with sampled-reference initialization and tracking alone' (Sec. V-C.b)
Teacher-free visual training
Exemplary deployment transparency
Failure-mode analysis with numbers
Weaknesses
Descriptive-only statistics
G1 manipulation is heavily simplified and sim-only
Unquantified per-task engineering
No stated code release, incomplete hyperparameters
Seed coverage ambiguous for Go2 obstacle tasks
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is SGPS and how does it train robot policies?
It combines sampling-based model-predictive control with first-order policy gradients through differentiable simulation. Sampling MPC generates a reference trajectory and recurring action targets; behavior cloning initializes the policy, and training alternates target refinement with short-horizon gradient-based tracking updates under perturbed states and randomized dynamics.
Does SGPS work on real robots or only in simulation?
For visual policies it avoids differentiating the renderer, so depth-based specialists train directly without a privileged state teacher. A DAgger-distilled unified Go2 policy transferred zero-shot to a real quadruped, autonomously trotting, crawling, and clearing hurdles from onboard depth. The humanoid G1 pushing and carrying results are simulation-only.
How is SGPS different from DIAL-MPC or MPPI?
DIAL-MPC is an online controller: it replans with sampling at runtime and needs state estimation (motion capture in its hardware tests). SGPS uses the same style of sampling offline to supervise policy training, then deploys a distilled neural policy with no online MPC and no reference inputs.
What compute is needed to train SGPS policies?
The paper trains all policies, including the refinement search, on a single NVIDIA RTX 4080 GPU using MuJoCo MJX, with a standard budget of about 4.1M gradient rollout interactions per task. Deployment runs on the Go2's onboard Jetson Orin.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- All quantitative results come from in-text approximations of plotted curves; figures themselves were not analyzable from the extracted text.
- Several quantities (acceptance thresholds κ and δ, tracking reward weights w_k, network sizes) are not stated in the extracted text; they may exist in an appendix that was not extracted.
- arXiv v1 preprint (Sep 2026); no peer-review verdict exists yet.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.