SE(3) Neural Potential Fields for 6-DoF Trajectory Planning Directly from Images Without Explicit 3D Reconstruction
Jeffrey Eiyike, Masoud Ataei, Elvis Gyaase, Vikas Dhiman
cs.RO · cs.AI
Quick verdict
基于导航函数监督的SE(3)神经势场可以从图像中规划6自由度抓取，仿真效果显著，但最终方法尚未在硬件上进行测试。
Summary
本文直接从桌面上 posed RGB 图像训练出用于末端执行器位姿的 SE(3) 神经势场。通过共享 MLP 将位姿扩展为三个刚性连接的夹爪点进行评估；NeRF 风格的投影使场的梯度与从实例分割导出的 2D 吸引/排斥场进行监督对齐，并以 AnyGrasp 抓取位姿作为额外的目标、量级和对齐损失的基准。由于图像导出的目标继承了人工势场的掠过和停滞失效问题，作者增加了导航函数监督：通过视觉外壳（训练视图的掩码膨胀 7 cm，在 1 cm 体素网格上进行 Dijkstra）到抓取点的测地距离，通过点采样损失蒸馏到网络中。在 Gazebo 中对 UR10 进行执行试验（两个场景，每个场景 10 个障碍物阻挡的起点），每条执行路径在真实几何结构下均无碰撞（而仅使用图像监督时分别为 25% 和 0%），平均间隙提升至 8.6–8.8 cm，机械臂连杆接触率降至 2.7–5.5%，规划耗时约 2 s，而基于 NeRF 重建的 RRT* 则需要 67–133 s。执行成功率为 90%（八个物体）和 40%（书本），归因于笛卡尔执行器的中止。局限性：仅两个场景且需按场景训练，最终的 SE(3) 方法无硬件执行，每个场景十个起点且未进行统计测试，且针对重建网格的离线无碰撞成功率仅为 45–100%。
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- An SE(3) neural potential field over end-effector poses can be trained from posed RGB images by projecting its gradient into views and matching segmentation-derived 2D potential gradients (Sec. IV-B).
- Distilling a navigation function — geodesic distance to the grasp through a 7 cm-inflated visual hull (Sec. IV-F) — removes grazing and stalling of image-supervised fields, yielding 100% collision-free executed paths versus 25% and 0% (Table I).
- Each 6-DoF reach plans in about 2 s versus 67–133 s for RRT* on a NeRF reconstruction of the same images, and the margin is the cost of dense collision checking, not planner complexity (Sec. VI-A/C).
- Residual execution failures (40% book-scene success) are attributable to the Cartesian executor, not the field, which converges within 3 cm on 10/10 starts (Sec. VI-D).
Strengths
Directly targets the two known failure modes of image-supervised potential fields
Sec. IV-F replaces grazing/stalling image-space targets with a navigation function (geodesic distance through a 7 cm-inflated visual hull) distilled at training time; Sec. VI-A quantifies the effect: CFGT rises from 0%/25% to 100% on both scenes, executed clearance from under 1 cm to 8.6–8.8 cm.
Fair, perception-matched baseline design
Sec. V-C gives RRT* the mesh the pipeline actually produced from the same views, with the ground-truth-geometry variant reported separately as an upper bound (Table I: 80%/80% SR); this isolates planner differences, unlike the authors' earlier evaluation that handed the baseline GT models.
Whole-arm safety auditing beyond TCP
Honest failure accounting
Ablations cover both supervision ingredients
Weaknesses
Tiny evaluation with no statistical treatment
Table II partially contradicts the safety narrative
Final method never executed on hardware
Navigation function quietly reintroduces offline geometry
Success metric is reach-not-grasp, and scope is per-scene
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper proposes an SE(3) neural potential field (NPF) for 6-DoF reach-to-grasp trajectory planning trained directly from posed RGB images, with no reconstruction extracted at planning time. Three claims anchor the work: (1) a 3D-to-2D projection (Sec. IV-B) can supervise a 3D pose-potential with segmentation-derived 2D attraction/repulsion fields; (2) distilling a navigation function — geodesic distance to the grasp through a 7 cm-inflated visual hull (Sec. IV-F) — removes the two classical artificial-potential-field failures (surface grazing, spurious-minimum stalling); (3) executed planning takes ~2 s versus 67–133 s for RRT* on a NeRF reconstruction of the same images, a margin the authors themselves attribute to dense collision checking rather than planner complexity. Claims are scoped honestly: success means reaching within 3 cm/15° of an AnyGrasp-proposed grasp pose, not object acquisition; hardware evidence covers only the earlier position-only formulation.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is an SE(3) neural potential field and how does this paper use it?
It is a neural network that assigns a scalar cost to any 6-DoF end-effector pose (position plus orientation); the robot moves by following the negative gradient, like rolling downhill. This paper trains it directly from posed RGB images: instance segmentation turns each camera view into 2D attraction (toward the target object) and repulsion (away from obstacles) fields, and a NeRF-style projection forces the 3D field's gradient to match them when rendered into those views. Three points rigidly attached to the gripper let position and orientation be planned jointly.
Why does the paper add a navigation function, and what problem does it fix?
Classical artificial potential fields fail when goal attraction and obstacle repulsion cancel: the robot grazes the obstacle instead of going around it, or stalls in a spurious minimum. During training only, the authors compute the geodesic (through-free-space) distance to the grasp on a coarse occupancy built from the image masks (a visual hull inflated by 7 cm) and distill it into the network. Because that distance decreases along every shortest path to the goal, it has no spurious minima, and the learned field inherits detouring behavior: executed paths became 100% collision-free versus 25% and 0% with image supervision alone.
Was the method tested on a real robot?
Not in its final form. The SE(3) navigation-function field is evaluated in executed Gazebo simulations on a UR10 (two scenes, 10 obstacle-blocked starts each). The real-robot rollouts shown use the authors' earlier position-only field without navigation-function supervision, and hardware trials of the final SE(3) method are explicitly listed as future work. The 40% success rate in one scene is attributed to the Cartesian motion executor refusing to start, not to the field itself.
How much faster is it than conventional planning, and what is the catch?
Once trained (about 6–7.4 hours per scene on one RTX 3090), a plan is gradient descent on the field and takes about 2 seconds, versus 67–133 seconds for RRT* planning on a NeRF reconstruction of the same images. The catch: the field is scene-specific (retrain for each new scene), and the paper itself argues the speed margin mostly reflects the cost of collision-checking a dense reconstruction rather than fundamental planner complexity — a cheaper geometric baseline on the same coarse hull was not tested.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Section VI-A contains a garbled/truncated fragment ('...entirely in collision avoidance.6-2.6 s)'), so some planning-time statements may be incomplete in the extraction.
- Figures (pipeline, trajectory renders) are only available via captions; visual evidence cannot be verified from text.
- Supplementary material (full hyperparameters and remaining ablations) is referenced but not included in the extracted text.
- The abstract in the metadata is truncated mid-sentence, though the full-text abstract is complete.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.