MarKey: Marginal Utility Guided Greedy Keyframe Selection for Long Video Understanding
Hongchang Shi, Jinpeng Hu, Ao Wang, Wenzheng Zhou, Hui Ma, Feng Li, Zenglin Shi
cs.CV · cs.CL
Quick verdict
Proposes greedy marginal-utility keyframe selection for long videos; solid gains on 4 of 6 benchmarks but trails DIG on two, with no significance tests.
Summary
The paper introduces MarKey, a training-free keyframe selection framework for long-video understanding with multimodal LLMs. Instead of scoring frames independently by query relevance, MarKey formulates selection as greedy maximization of a subset-conditioned marginal utility combining three terms: CLIP-based query relevance, a marginal coverage gain estimated against a compact anchor subset (648 frames from relevance-stratified clip sampling), and a redundancy penalty computed against a bounded sliding window (W=28) of previously selected frames, yielding O(KN(A+W)) complexity. With Qwen3-VL-8B at 32 frames, MarKey reaches 65.74% on LongVideoBench, 68.14% on Video-MME, 37.62% on EgoLifeQA, 84.20% on NExT-QA, CIDEr 48.90 on YouCook2, and 29.70 on Video-TT, with gains confirmed on Qwen2.5-VL-7B and MiniCPM-V-4.5-8B, across 2B-32B scales and 16-128 frame budgets. Ablations attribute the largest share to query relevance. However, 'consistent' superiority is overstated: DIG outperforms MarKey on NExT-QA (84.61 vs 84.20) and Video-TT (32.10 vs 29.70), also edging the open-ended average. No significance tests, seeds, or variances are reported; hyperparameters are argmax on the evaluation benchmarks themselves; the MPCS redundancy metric uses the same CLIP encoder as the selection objective; no code release or wall-clock overhead is reported.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Evaluating candidates relative to the already-selected subset (context-aware marginal utility) yields better keyframe sets than independent relevance scoring under a fixed frame budget.
- A surrogate utility combining CLIP query relevance (alpha=0.7), anchor-based marginal coverage gain (delta=0.2), and a sliding-window redundancy penalty (lambda=0.1) approximates subset-utility greedy selection at O(KN(A+W)) cost.
- With Qwen3-VL-8B at 32 frames, MarKey achieves the best results among compared training-free selectors on LongVideoBench (65.74%), Video-MME (68.14%), EgoLifeQA (37.62%) and YouCook2 (48.90 CIDEr).
- Gains transfer across backbones (Qwen2.5-VL-7B, MiniCPM-V-4.5-8B), model scales (2B-32B), frame budgets (16-128), and grow with video duration (up to +7.8 points on videos longer than 20 minutes).
- Lower selected-frame redundancy (MPCS 0.6177/0.6773 on LVB/V-MME) co-occurs with higher downstream accuracy, supporting the complementarity hypothesis.
Strengths
Well-motivated subset-aware formulation with a clean mechanism control
Eqs. 2-3 recast selection as marginal contribution beyond the chosen set; the matched Top-K baseline (same CLIP-L/14 scorer, §4.1/§4.2) isolates the subset effect, showing +3.59 (LVB) and +3.73 (V-MME) points over pure relevance ranking.
Scalable approximations with explicit complexity analysis
Anchor-based coverage (A=648, §3.3) and a W=28 sliding window (§3.3) reduce O(KN^2)+O(NK^2) to O(KN(A+W)) (§3.4, Eq. 16); Fig. 6 sweeps show accuracy is flat near the defaults (776 anchors slightly hurts), so the approximations are not brittle.
Broad, layered empirical coverage
Component ablation cleanly attributes gains
Weaknesses
Headline claim contradicted by the paper's own tables
No statistical rigor
Hyperparameters evaluated on the test benchmarks
Efficiency motivation never measured
Conceptual overlap and self-referential redundancy metric
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
MarKey (arXiv 2609.15408; Hefei University of Technology) is a training-free keyframe-selection framework placed before a frozen MLLM for long-video QA. The central claim: frame selection should be optimized subset-conditionally, scoring each candidate by an approximate marginal utility combining CLIP query relevance (alpha), marginal coverage gain over representative anchors (delta), and a redundancy penalty against previously selected frames (lambda). Supporting claims: state-of-the-art results among training-free selectors on six benchmarks; robustness across three MLLM backbones, scales 2B-32B, and budgets 16-128 frames; advantages that grow with video duration. The motivating diagnosis (§1, Fig. 1) — individually relevant frames can be collectively redundant — is well taken and directly testable via the paper's own Top-K control.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is MarKey in one sentence?
MarKey is a training-free keyframe selection method for long-video multimodal LLMs that greedily picks frames maximizing a marginal utility combining CLIP query relevance, coverage gain over 648 anchor frames, and a redundancy penalty against a sliding window of already-selected frames.
Does MarKey beat all other keyframe selection methods?
No. With Qwen3-VL-8B at 32 frames it is best among the compared training-free selectors on LongVideoBench (65.74%), Video-MME (68.14%), EgoLifeQA (37.62%) and YouCook2 (48.90 CIDEr), but DIG scores higher on NExT-QA (84.61 vs 84.20) and Video-TT (32.10 vs 29.70); MarKey wins the holistic and human-centric category averages while DIG edges the open-ended average (39.49 vs 39.30).
Does MarKey need training or extra models?
No training is required. It uses an off-the-shelf CLIP-L/14 to embed frames and the query, and adds an inference-time selection step before a frozen MLLM; hyperparameters are alpha=0.7, delta=0.2, lambda=0.1, up to 648 anchors, and a context window of 28.
Is code for MarKey available?
The paper does not mention a code release. The method is fully specified by Algorithm 1 and the stated hyperparameters, but reproducing headline numbers would require reimplementing the selection loop and re-running LMMS-Eval evaluations on public benchmarks with a Qwen3-VL-8B-class model.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-8 are available only through embedded numeric labels and captions; visual content (Fig. 2 pipeline diagram, Fig. 8 case-study frames) could not be independently verified.
- Baseline numbers for AKS/BOLT/FOCUS/DIG/AIR are as run by the authors; fairness of their reimplementations and prompt setups cannot be checked from the text.
- No supplementary material, code link, or project page appears in the extracted text; availability statements may exist outside the main body.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.