HAP: A Hand-Driven Active Perception Framework for Egocentric Head Motion Prediction
Yunji Feng, Junyi Ma, Guanzhong Sun, Chenyang Xu, Hesheng Wang
cs.CV · cs.RO
Quick verdict
HAP predicts egocentric 6-DoF head motion from hand-inferred intent and occlusion graphs; credible ablations, but gains over a constant-velocity prior are thin.
Summary
The paper formulates egocentric 6-DoF head (camera-pose) motion prediction during object manipulation as an active-perception problem conditioned on observed hand motion and the inferred interaction target. HAP estimates a soft distribution over candidate targets from 3D hand positions (WiLoR) and hand-object geometry, then builds a dynamic Predictive Target-Centric Amodal Occlusion Graph (P-TAOG): object nodes carry target probabilities, directed edges encode current (z-buffered point-level) and potential (offset-projected) occlusion, and the graph is encoded by directed PNA layers plus a causal GRU. A horizon-wise gate blends the learned trajectory with a constant-velocity SE(3) prior, trained with horizon-weighted Smooth-L1 loss. The authors also introduce Bottle, a 382-clip egocentric RGB-D reach-to-target dataset (12 sessions, 10/1/1 split). On Bottle, HAP beats 12 baselines including CVH, UniHand, and diffusion variants on all four metrics (tADE 18.56 mm vs 26.09 mm for RVAE); on EgoPAT3Dv2 it is best in translation (10.56 mm tADE, near CVH's 10.69) and second in rotation behind RVAE. Ablations show the gate helps everywhere while ungated P-TAOG underperforms motion-only, and the full design wins rotation but not tADE/tFDE among design variants. Main limitations: a single test session, no variance or significance testing, unclear input parity for baselines, and a heavy multi-model preprocessing pipeline.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Future egocentric 6-DoF head motion during manipulation is driven by hand-inferred target intent and evolving target visibility, and can be predicted from observed hand-head history.
- Modeling current and potential target occlusion in a dynamic directed graph (P-TAOG), fused with hand/head motion history, lowers head-motion prediction error relative to motion-history-only predictors.
- A horizon-wise gate blending a constant-velocity SE(3) prior improves both motion-only and P-TAOG-conditioned variants (Table II).
- HAP achieves the best results among evaluated methods on Bottle (all four metrics) and the best translation / second-best rotation errors on EgoPAT3Dv2 (Table I).
- Preserving a soft target distribution outperforms hard target assignment, and intention is anticipatory: 92.79% pre-contact top-1 accuracy on curated EgoPAT3Dv2 validation (Sec. IV-E).
Strengths
Novel, well-motivated problem framing
Reframes head motion as active perception conditioned on hand-inferred intent and target visibility (Sec. I), going beyond Uni-Hand's auxiliary head branch; the 6-DoF formulation with SE(3) relative transforms and continuous 6D rotation representation (Eqs. 1-2) is clean and standard.
Explicit, interpretable occlusion modeling
P-TAOG (Sec. III-C) grounds amodal occlusion in z-buffered point-level reasoning over completed geometry (Eq. 10) and adds a potential-occlusion term from projected offsets (rpot) — a concrete, inspectable mechanism rather than an implicit latent.
New targeted dataset with changing visibility
Candid factorial ablations and backend robustness check
Weaknesses
Thin margins over a trivial prior on the public dataset
No uncertainty quantification; single test session
Contribution of P-TAOG is conditional and mixed
Possible input-modality confound against baselines
Heavy, partially undocumented pipeline
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is HAP in egocentric head motion prediction?
HAP (Hand-driven Active Perception) is a framework that predicts a person's future 6-DoF head motion — position and orientation over a forecast horizon — from a head-mounted RGB-D camera. It first infers which object the user is reaching for from 3D hand motion, then builds a directed occlusion graph (P-TAOG) tracking how scene objects block the target now and could block it after small viewpoint changes, and finally blends a learned trajectory with a constant-velocity prior through a per-horizon gate.
What is the Bottle dataset?
Bottle is an egocentric RGB-D dataset introduced with the HAP paper: 382 clips of reach-to-target object manipulation in cluttered scenes, recorded across 12 sessions (10 for training, 1 validation, 1 test), featuring varied object layouts and frequent changes in target visibility. Head motion is represented by the head-mounted camera trajectory, estimated and smoothed with Depth Anything 3.
How well does HAP perform compared to baselines?
On the Bottle dataset, HAP reports the lowest errors on all four metrics, for example an average translation error of 18.56 mm versus 26.09 mm for the best prior method (RVAE). On the public EgoPAT3Dv2 dataset it achieves the best translation errors (average 10.56 mm, close to the 10.69 mm of a simple constant-velocity heuristic) and the second-best rotation errors behind RVAE. All results are single-run point estimates without variance reporting.
Is code or data available for HAP?
The paper states that code will be released at the project page (HAP-ego.github.io/HAP). Release terms for the Bottle dataset are not stated in the paper text, and the pipeline depends on several third-party models (YOLO11, SAM 2, WiLoR, SAM 3D Objects, and Depth Anything 3 or ORB-SLAM3 for camera tracking).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-6 could not be inspected from the text, so qualitative claims in Sec. IV-C (Fig. 6 trajectory overlays) are accepted as stated.
- Observation window length and prediction horizon H are not given in the extracted text, limiting reproduction checks.
- Supplementary material (dynamic visualizations) is referenced but not included in the extracted text.
- Text extraction renders some equations subscripts approximately; method details were cross-checked against the surrounding prose.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.