Touch2Robot: Robot Touch in the Human Demonstration Loop
Shengcheng Luo, Xiaoyang Cheng, Hong Ying, Xiaoying Zhou, Jiaming Jiang, Haoran Guo, Wanlin Li, Ziyuan Jiao, et al.
cs.RO · cs.AI
Quick verdict
Simulated robot-touch feedback in human demos nearly doubles replay success (37.9→72.1%) and lifts downstream policy; strong real-robot study, one embodiment.
Summary
Touch2Robot is a data-collection framework that lets humans demonstrate dexterous manipulation while a VR headset shows, in real time, where the target robot hand would make contact with the object. Human hand motion (MANUS glove), tactile recordings (Juqiao glove), and object pose (FoundationPose) train object-specific RL teachers in MuJoCo that reproduce demonstrated object motion while matching the human's contact pattern; these teachers are distilled into a single geometry-conditioned Transformer retargeter running at about 1.1 ms per frame. Reconstructed contacts are visualized at 30 Hz (122 ms latency) so demonstrators can adjust their actions online. On four real-robot tasks with a LEAP Hand on an xArm6, average replay completion rises from 37.9% with visual-only feedback to 72.1%, and time per replay-successful demonstration drops from 58.6 s to 18.2 s. A Diffusion Policy trained on these demos reaches 62.0% average completion versus 32.9% for visual-only feedback. Reconstructed contacts match real tactile measurements at 44.2% F1 and human preferences at 47.5% F1 — better than the ConTrack and DexMachina baselines but still modest, with higher false-positive rates. Main limitations: a single robot embodiment, four tasks, no statistical tests for the main tables, an unstated operator count, and only qualitative cross-hand results.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Visualizing reconstructed target-hand contacts during human demonstration collection nearly doubles average real-robot replay completion (37.9% → 72.1%) and cuts time per replay-successful demonstration from 58.6 s to 18.2 s (Table I).
- Treating human touch as a cross-embodiment contact preference yields reconstructed contacts with 44.2% F1 against real LEAP-Hand tactile measurements and 47.5% F1 against human preference, outperforming ConTrack, DexMachina and a no-tactile-reward ablation (Tables III–IV).
- Diffusion Policy trained on Touch2Robot demonstrations reaches 62.0% average completion across four tasks, +29.1 percentage points over visual-only collection (Table II).
- A single distilled geometry-conditioned retargeter retains most teacher performance (contact F1 65.46 vs 73.70) at 1.12 ms inference latency, enabling real-time closed-loop collection (Table VI).
Strengths
Closed-loop robot-contact feedback without robot execution
The core idea is genuinely new in this combination: simulator-reconstructed LEAP-Hand contacts are shown to the demonstrator on a Quest 3 at 30 Hz with 122 ms latency (Sec. III-C), going beyond ARCap/ARMADA-style motion-only overlays; Table I shows this feedback, not the retargeter alone, drives the 37.9%→72.1% replay gain.
Layered, honest evaluation
Five complementary evaluations: replay (Table I), downstream Diffusion Policy (Table II), fidelity vs real tactile (Table III, 44.19% F1), preference alignment (Table IV, 47.53% F1), and a properly powered user study (Table V, Wilcoxon signed-rank with Holm correction, p<0.01). The paper openly reports its modest F1 and the FPR increase (6.38% vs 2.01% for the no-tactile ablation) instead of hiding them.
Downstream transfer is demonstrated, not assumed
Unusually detailed supplementary material
Weaknesses
Modest reconstruction fidelity undermines the feedback signal
Thin statistical treatment of headline tables
Single embodiment, four objects, no unseen-object test
Scalability claim is an argument, not a measurement
Gains are task-dependent but reported as averages
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Touch2Robot (arXiv 2609.24660v2, cs.RO) is a system paper for human demonstration collection in dexterous manipulation. Its thesis: human demos scale well but their contacts transfer poorly to robot hands, and the missing ingredient is showing the demonstrator the *target hand's predicted contact state* online. The framework captures human hand motion (MANUS glove), tactile readings (Juqiao glove), and object 6D pose (FoundationPose); trains per-object PPO teachers in MuJoCo to reproduce demonstrated object motion under a binary semantic-contact reward matching human touch (Sec. III-A); distills all teachers into one geometry-conditioned causal-Transformer retargeter (Sec. III-B); and visualizes simulated robot-object contacts on a Quest 3 at 30 Hz with 122 ms latency (Sec. III-C). Central claims: (1) replay completion 37.9%→72.1% vs visual-only feedback and collection time per replay-successful demo 58.6 s→18.2 s (Table I); (2) reconstructed contacts at 44.19% F1 vs real LEAP tactile and 47.53% F1 vs human preference (Tables III–IV); (3) downstream Diffusion Policy at 62.0% vs 32.9% average completion (Table II); (4) the unified retargeter keeps contact F1 65.46 vs the teachers' 73.70 at 1.12 ms/frame (Table VI).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Touch2Robot and what problem does it solve?
Touch2Robot is a data-collection framework for dexterous robot manipulation. When humans collect demonstrations, their contact patterns often fail to transfer to the robot hand. Touch2Robot shows the demonstrator, on a VR headset and in real time, where the target robot hand (a LEAP Hand) would touch the object — predicted via a learned retargeter plus physics simulation — so the human adapts their motions before data ever reaches the robot. No physical robot is used during collection.
How much better is Touch2Robot than existing demonstration collection methods?
On four real tasks with an xArm6 arm and LEAP Hand, average real-robot replay completion rose from 37.9% with visual-only VR feedback to 72.1% with Touch2Robot (offline retargeting baseline: 28.3%). Time to collect one replay-successful demonstration fell from 58.6 s to 18.2 s, and a Diffusion Policy trained on the collected data improved from 32.9% to 62.0% average task completion.
How accurate is the simulated contact prediction?
Reconstructed contacts matched the real robot's tactile measurements with 44.2% F1 and the human demonstrator's own touch with 47.5% F1, with contact-onset errors of 857 ms and 292 ms respectively. This outperforms interaction-aware baselines (ConTrack at 23.8%, DexMachina at 24.7% F1) but is far from perfect, and comes with a higher false-positive rate (~6.4%).
Does collecting data with Touch2Robot require a real robot?
No. During collection, the robot hand and its predicted contacts are computed in simulation and displayed on a Meta Quest 3 at 30 Hz with 122 ms end-to-end latency. The physical robot is used only afterwards, to replay the retargeted demonstrations and to evaluate policies trained on them.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures (Figs. 1-13) are not analyzable from extracted text; the pairwise agreement values in Fig. 7 and reward curves in Fig. 8 cannot be numerically verified.
- The number of demonstration operators behind Tables I-II and the evaluation trial counts for Diffusion Policy are not stated anywhere in the extracted text.
- Author metadata differs slightly from the paper text: the text lists Chenxi Xiao as corresponding author while the provided metadata list omits him, and the second author appears as 'Cheng' in metadata but 'Chen' in the paper.
- arXiv v2 (22 Sep 2026) cites many 2026 preprints (ReForce, TopoRetarget, ConTrack); novelty relative to these contemporaneous works could not be verified offline.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.