Finder: Agentic Closed-Loop Object Finding for Embodied Grounding
Shixiong Xu, Zhiyuan Chen, Song Ding, Rui Luo, Xiaowei Liang, Dongxu Miao, Zhiying Du
cs.CV
Quick verdict
Agentic closed-loop object grounding with +15.75 S@1 over baselines; promising, but the single-shot ablation suggests SAM3 perception drives much of the gain.
Summary
The paper introduces Finder, an agentic closed-loop primitive for open-vocabulary object finding in partially observed 3D scenes. Rather than one-shot retrieval over a pre-built scene graph, Finder organizes each grounding episode as a five-stage loop (plan, detect, selection plan, select, judge) over a typed protocol state: phrase-conditioned SAM3 detections are backprojected into 3D candidates, candidate-centric evidence cards are ranked by a selection planner, a VLM selector performs factual visual verification, and an LLM/VLM judge alone decides accept, continue, or abort, emitting structured hints that redirect subsequent perception. An outer harness injects task- and session-level context, enabling transfer to sequential grounding (SG3D) and object-centric QA (OC-NaVQA) without changing the loop. On a self-constructed 1,632-query benchmark spanning nine HM3D scenes and four real RGB-D environments, Finder reports 66.24% average S@1 versus 50.49% for the strongest baseline (HOV-SG); on SG3D it gains +3.75 s-acc over DAAAM+GPT, and on OC-NaVQA it achieves better localization errors though QA accuracy (0.540) trails DAAAM (0.711). Ablations attribute gains to support-aware verification and judge feedback, but the Single-shot variant retains 63.55% S@1 with lower localization error than the full system. Limitations include dependence on pre-captured posed RGB-D streams, proprietary API models, absent significance testing, a small real-world split, and 80-96 s per-query runtimes without baseline cost comparisons.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Reframing open-vocabulary object grounding as an agentic closed loop with typed state and explicit accept/continue/abort control improves retrieval, yielding +15.75 average S@1 points over strong baselines on Habitat/HM3D and real-world RGB-D scenes.
- A judge-only stopping boundary, combining visual facts, support-relation metrics, room binding, and loop history, turns retries into structured follow-up rather than blind repetition.
- Support-object evidence gathered with adaptive execution scopes is necessary for relational queries, as removing it degrades performance in the Table 5 ablation.
- The same inner protocol transfers to sequential grounding (SG3D) and object-centric QA (OC-NaVQA) purely through context injection by an outer harness, without modifying the loop.
Strengths
Clean protocol formalization with explicit separation of concerns
Section 3.1 defines the loop as typed equations (1)-(5) with stable state objects (FinderRequest, SearchPlan, LoopContext, LoopState); restricting termination authority to the judge while keeping the VLM selector factual (Sec 3.3, prompts D.2-D.3) is a well-motivated design that makes retries inspectable and modules swappable.
Unusually thorough ablation and sensitivity suite
Table 5 isolates every loop mechanism (feedback, support targets, selector, judge, support scope); Table 7 varies planner/VLM API families (S@1 64.71-68.69); Figure 3 sweeps frame budget, stride, and phrase counts; Table 13 reports runtime by object type (80.5-96.1 s).
Transparent experimental artifacts
Honest negative reporting
Weaknesses
Detector-versus-loop confound undercuts the headline claim
No statistical rigor anywhere
Success metric does not reward the disambiguation the paper motivates
Efficiency claims unquantified against baselines
Thin real-world evidence and opaque baseline adaptation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Finder and what problem does it solve?
Finder is a system from Xiaomi Robotics that finds the object referred to by a language query (e.g., the chair near the table) in a partially observed 3D scene. Instead of one-shot retrieval from a pre-built 3D map, it runs a closed loop of planning, detection, candidate selection, and verification, and explicitly decides whether to accept a candidate, gather more evidence, or give up.
How much better is Finder than previous methods?
On the paper's own 1,632-query benchmark spanning 9 simulated HM3D scenes and 4 real indoor environments, Finder reports 66.24% average Success@1m versus about 50% for the strongest adapted baselines such as HOV-SG, a 15.75-point average improvement. Gains are much smaller on sequential grounding (SG3D) and its question-answering accuracy on OC-NaVQA is lower than the DAAAM baseline, though its localization errors are better.
Is code or data available for Finder?
The paper points to a project page but does not state a code or dataset release. It does publish full prompt templates, default hyperparameters, and per-scene results. Reproduction currently requires proprietary API models (MiMo-v2-Pro and Qwen family) and the SAM3 segmenter.
What are the main limitations of Finder?
It assumes pre-captured RGB-D video with known camera poses rather than active robot exploration; it cannot aggregate over multiple objects (e.g., counting); it needs roughly 80-96 seconds per query; and the real-world evaluation covers only 82 queries in 4 scenes. Its own ablation shows a single-shot version achieves most of the accuracy, so the closed loop adds moderate rather than dramatic gains.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 are only partially recoverable from extracted text; the budget-study values in Figure 3 appear inline but axis semantics could not be fully verified.
- Baseline numbers for SG3D and OC-NaVQA are cited from prior papers and could not be independently verified.
- Review is based on arXiv v1 (2609.18058); peer-review status and any camera-ready changes are unknown.
- Whether code or the benchmark will be released was judged from the paper text only; the project page was not consulted.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.