Companion-style QA Assistance in Ego-Vision
Hangyu Qin, Junbin Xiao, Shenglang Zhang, Angela Yao
cs.CV · cs.AI
Quick verdict
Presents BuddyVQA (21.6K deictic, chained egocentric-stream QA) plus MyBuddy; gains look real, but LLM-judge design and an EgoSchema anomaly need scrutiny.
Summary
The paper introduces BuddyVQA, a benchmark of 21,574 questions over 6,002 highlight moments in 1,012 EgoSchema-derived egocentric videos (~50 hours), targeting companion-style QA where questions contain ego-deictic references (87.6%) and interactive chains (79.1%). Questions were drafted by Gemini-2.5-Pro, rewritten by 25 human annotators under streaming constraints, and carry 2.76 reference answers on average after entailment-based dedup. The authors propose MyBuddy, a training-free framework combining multi-level heterogeneous memory (frame embeddings, incremental DBSCAN clusters, idle-time summaries), a question filter retrieving relevant historical QAs, gaze/hand ego-cue extraction, and hierarchical question rephrasing. On BuddyVQA, MyBuddy reaches 65.1% accuracy with a GPT-5.5 backbone versus 54.6% for the strongest streaming baseline (ReKV), with gains on chained (62.8% vs 50.9%) and deictic (63.5% vs 48.6%) subsets; it also improves RVS-Ego/RVS-Movie and Video-MME results and cuts latency and GPU memory versus naive context concatenation. Main limitations: all accuracy comes from a GPT-5 judge whose alignment with humans was validated only on two baselines, not on MyBuddy or the commercial backbones being judged; the deictic statistics are partly induced by generation prompts that require pronouns; and MyBuddy(Qwen3.5) drops to 47.8% on EgoSchema versus its backbone's 68.1%, unexplained in the provided text.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- BuddyVQA is the first benchmark to combine ego-deictic expressions and interactively chained questions under streaming, egocentric, open-ended QA (Table 1).
- Existing general and streaming MLLMs score at most ~55% on BuddyVQA (best baseline ReKV 54.6%), exposing failures in referential grounding and history use.
- MyBuddy, a training-free framework, improves three backbones to 59.9-65.1%, with the largest margins on chained (+11.9) and deictic (+14.9) questions over ReKV.
- The multi-level memory and question filter improve efficiency, cutting latency from 3.9s to 2.6s and GPU memory from 51.1GB to 29.1GB versus naive context concatenation (Table 8).
- Gains generalize to RVS-Ego (66.1%), RVS-Movie (58.9%), and Video-MME/EgoSchema for the commercial backbones.
Strengths
Genuinely new benchmark axis with a clear positioning table
Table 1 shows BuddyVQA is the only dataset combining deictic expressions (87.6%), chained questions, egocentric videos, multi-answer GT, question categories, and open-ended evaluation; nearest neighbors EgoBlind and EgoPointVQA lack chaining and cover narrower populations/protocols.
Credible multi-stage annotation pipeline with design validation
25 Prolific annotators rewrite Gemini-2.5-Pro drafts under a no-future-frames constraint, must self-author >=50% of QAs, and subjective questions get >=3 answers deduplicated via roberta-large-mnli at 0.75 similarity (mean 2.76 answers/question). A human study supports the core premise: 57.6% of chained questions are unanswerable without base QAs.
Large, consistent gains with component-level ablations
Efficiency claims are backed by measured numbers
Diagnostic depth beyond headline scores
Weaknesses
Judge-backbone family overlap creates self-preference risk
Unexplained EgoSchema degradation undercuts the generalization claim
Signature statistics are partly elicited by the generation prompt
No uncertainty quantification anywhere
Sampling-protocol asymmetry vs the strongest commercial baseline
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a dual-contribution paper: (1) **BuddyVQA**, a streaming egocentric VideoQA benchmark built from 1,012 EgoSchema videos (704 test / 308 validation; ~50 h total) with 21,574 questions over 6,002 highlight moments; and (2) **MyBuddy**, a training-free QA framework. The central claims: existing benchmarks miss ego-deictic references and interactively chained questions (Table 1 positions BuddyVQA as the only dataset combining deictics + chaining + egocentric video + multi-answer + open-ended evaluation); current models score below ~55% (best baseline ReKV at 54.6%, Table 2); and MyBuddy lifts three backbones to 59.9-65.1% while generalizing to streaming and offline benchmarks.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is BuddyVQA?
BuddyVQA is a video question answering benchmark of 21,574 questions over 6,002 highlight moments in 1,012 long egocentric videos (about 50 hours, sourced from EgoSchema). It simulates a first-person 'companion' assistant: questions use deictic pronouns like 'Where is it?' (87.6% of questions) and depend on earlier question-answer turns (79.1%), and models must answer using only video seen before each question's timestamp.
What is MyBuddy and how much does it improve accuracy?
MyBuddy is a training-free framework placed on top of an existing vision-language model. It adds a three-level memory (recent frame features, clustered mid-term features, long-term text summaries), a filter that retrieves relevant past QAs, gaze/hand 'ego-cue' extraction, and question rephrasing to resolve vague references. On BuddyVQA it reaches 65.1% accuracy with GPT-5.5, versus 54.6% for the strongest prior streaming baseline (ReKV).
How is BuddyVQA different from EgoSchema or StreamingBench?
Those benchmarks ask mostly self-contained single questions. BuddyVQA uniquely combines deictic references, chained questions that require earlier QA turns (a human study found 57.6% of them unanswerable without the base QA), multiple reference answers per subjective question, and timestamped streaming queries over long first-person video.
Is the BuddyVQA dataset or MyBuddy code available?
The paper states that code and dataset are available at github.com/QHUni/BuddyVQA, and the data availability statement commits to public release of BuddyVQA and MyBuddy with access instructions for reproducibility. Note that headline results use commercial models (GPT-5.5, Gemini-3.1-pro) that require paid API access.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with five '[...sampling gap...]' regions (parts of dataset-construction details, the evaluation setup around Section 5.2, case-study figure text, references, and part of the Table 16 filtering prompt); every number cited in this review comes from visible text only.
- The count of first-round discarded questions falls inside a sampling gap, so the annotation funnel before the ~29K second-round figure could not be verified.
- Whether the MyBuddy(Qwen3.5) EgoSchema drop is discussed elsewhere (e.g., inside a sampling gap or appendix) could not be confirmed; it is described here as unexplained in the provided text, not as absent from the paper.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.