VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understanding
Weixin Xu, Zhenyu Yang, Bing Wang, Shengsheng Qian, Changsheng Xu
cs.CV · cs.AI
Quick verdict
Trained 7B agent learns adaptive pacing for long-video QA via SFT+DAPO RL; strong open-source gains and thorough ablations, but no significance tests or code.
Summary
The paper reframes long-video question answering as Sequential Evidence Acquisition (SEA): a multi-turn agent traverses the video along the temporal axis, choosing at each turn a viewing speed (1x/2x/4x), updating a bounded textual memory, optionally rewatching a recently fast-forwarded chunk, or answering. The authors train VideoScout-7B on Qwen2.5-VL-7B in two stages: turn-level SFT on VideoScout-66K (66K turns distilled from ~10K answer-verified Gemini 3 Flash trajectories via rejection sampling), then trajectory-level DAPO with a composite reward (accuracy, format, and teacher-timing IoU gated on correctness). On nine benchmarks it reports the best open-source 7B averages (48.0 on four long-video benchmarks, 45.0 on five cross-domain ones), beating its base model by 6.2/7.2 points and an untrained agentic variant by 14.9/10.7. Ablations credit both training stages, all three reward terms, and the notes/rewatch mechanisms; inference time adapts from 8.1s (WorldSense) to 35.1s (LVBench). Main limitations: no significance testing or seed variance, unquantified train-eval contamination risk, inference cost up to ~3x agentic baselines on long videos, and no code or data release.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Long-video QA under a bounded visual context is effectively a turn-by-turn Sequential Evidence Acquisition problem, solvable by a trained agent that controls viewing speed, keeps textual notes, and rewatches skimmed chunks.
- Two-stage training — turn-level SFT on 66K distilled turns followed by trajectory-level DAPO with an accuracy+format+timing-IoU reward — is essential: the same agentic framework without training scores below the base model (33.1 vs 41.8 average, Table III).
- VideoScout-7B achieves the best open-source 7B averages across nine benchmarks: 48.0 on four long-video benchmarks (Table III) and 45.0 on five cross-domain benchmarks (Table IV).
- The learned pacing is adaptive rather than fixed: turns and inference time scale with video length and difficulty (8.1s on WorldSense to 35.1s on LVBench), and accuracy does not degrade on videos longer than 30 minutes (Fig. 8).
Strengths
Untrained agentic control isolates the training effect
Qwen2.5-VL-7B‡ runs the identical multi-turn framework but scores 33.1 avg vs the base model's 41.8 on Table III (34.3 vs 37.8 on Table IV); Fig. 9 shows the untrained variant stays at 1× and answers within a few turns — direct evidence that gains come from two-stage training, not from the scaffold.
Unusually complete ablation program
Tables VI-VIII cover training stages, GRPO-vs-DAPO, all three reward terms, frame budgets (re-trained under the alternative budget, not merely re-evaluated), chunk duration, speed policy, and both core mechanisms: w/o Notes costs 9.0 on MLVU (51.0→42.0), w/o rewatch 2.6 on LVBench (45.1→42.5).
Adaptive compute is demonstrated, not asserted
Precise agent and environment specification
Honest failure analysis
Weaknesses
No variance or significance reporting anywhere
Train-eval contamination surface unquantified
Teacher-coupled behavior and reward
Compute fairness gaps in comparisons
No code, data, or model release; proprietary teacher
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets long-video question answering when decisive evidence is sparse and the video far exceeds the model's per-turn visual budget (a 2-hour film at 1 fps exceeds 7B-model context by an order of magnitude, Sec. I). It makes four claims: (1) long-video QA under bounded visual context is best cast as Sequential Evidence Acquisition (SEA), a turn-by-turn temporal reading process; (2) the VideoScout agent — adaptive speed (1×/2×/4×), bounded textual notes, and a local rewatch action — realizes SEA; (3) a two-stage pipeline (turn-level SFT on VideoScout-66K, then trajectory-level DAPO with an accuracy + format + timing-IoU reward) trains this behavior on Qwen2.5-VL-7B; (4) the result is the strongest open-source 7B on nine benchmarks, averaging 48.0 on four long-video benchmarks (Table III) and 45.0 on five cross-domain benchmarks (Table IV).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is VideoScout?
VideoScout is a 7B multimodal agent for long-video question answering. Instead of sampling a fixed frame set once, it watches the video turn by turn: at each turn it chooses a viewing speed (1×, or 2×/4× fast-forward), maintains a short textual memory of what it has seen, can rewatch a recently fast-forwarded segment at full detail, and decides when it has gathered enough evidence to answer.
How does it differ from coarse-to-fine zooming methods like LongVT?
Zooming methods first build a sparse global overview and then zoom into windows selected from that overview. On very long videos the overview itself can miss the decisive moment, so the zoom lands in the wrong interval. VideoScout scans forward sequentially at adaptive speed, so it never depends on a one-shot global scan, and it can rewatch skimmed chunks at 1× to recover fine detail.
What results does the paper report?
Trained from Qwen2.5-VL-7B, VideoScout reports the best open-source 7B averages across nine benchmarks: 48.0 over LVBench, MINERVA, MLVU and Video-MME, and 45.0 over VideoMMMU, SciVideoBench, WorldSense, VSI-Bench and MMVU. This is +6.2/+7.2 points over its base model and +14.9/+10.7 over the same multi-turn framework left untrained. Inference time adapts with difficulty, from 8.1 seconds on WorldSense to 35.1 seconds on LVBench.
What are VideoScout's main limitations?
Per the paper: its textual notes can propagate an early misreading across a long rollout; the rewatch action only reaches chunks inside the most recent fast-forward window, so early segments cannot be revisited (a failure case is shown); it takes no audio input; and no code or data had been released with this preprint.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1, 2, 6, 8, 9 are present only as captions — curve/bar values (duration-bucket accuracy, reward trends, action distributions) could not be verified from text.
- Data-distribution details are internally inconsistent in the extraction: the prose says 1% of SFT samples span 10-30 minutes while Fig. 5(a) labels show different values; treat the SFT/RL duration splits as uncertain.
- This is a v1 preprint extraction; no supplementary material, the full system prompt, or per-benchmark sample counts were available.
- Some baseline rows are partially empty (e.g., Video-MTR-7B reports only LVBench and MLVU), limiting like-for-like comparison.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.