Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs
Killian Steunou, Yannis Tevissen, Mounîm A. El Yacoubi
cs.CV · cs.CL · cs.MM
Quick verdict
Survey of 125 works mapping VideoLLM efficiency by pipeline stage; disciplined comparisons, but evidence rests on borrowed re-runs, not new experiments.
Summary
This survey organizes inference-efficiency mechanisms for VideoLLMs by the pipeline stage where they act: input construction (frame sampling, resolution), encoder computation, connector-level token reduction, and LLM execution (pruning, sparse attention, summary tokens, KV-cache management). From several hundred candidates, 125 papers (through August 2026) were retained via keyword search and citation snowballing; a GitHub repository is maintained. Section III-B formalizes FLOPs and KV-memory scaling across the encoder-connector-LLM pipeline. The methodological signature is comparison discipline: controlled shared-host tables (frame samplers on LLaVA-Video-7B, Table II; HoliTom's re-runs on LLaVA-OneVision-7B, Table V; HieraVid's re-runs on LLaVA-Video-7B, Table VII; streaming systems, Table VI) are separated from indicative cross-paper evidence (Table IV). Key findings: retaining ~25% of visual tokens often preserves near-baseline accuracy, but at 10% retention temporal-aware methods (PruneVid, HoliTom) degrade far more gracefully than spatial-only ones (VisionZip, average falls to 91.6%); selection is not always upstream, as Frame-Voyager reports 27.6% higher latency than uniform sampling; and the strongest 2025-2026 methods combine stages. Main limitations: no experiments of its own, controlled comparisons borrowed from competing method authors' re-runs, thin audiovisual coverage (essentially OmniZip and DASH), and conclusions anchored to 7B-8B hosts on MCQ benchmarks.
Review complete
Assessment at a glance
3
risk flags identified
● high 0
● medium 2
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Retaining roughly one quarter of the visual-token budget often preserves near-baseline accuracy, but method choice becomes decisive at 10% retention (Sec. IV-D, Tables V/VII).
- Selection is not always upstream: query-conditioned selectors that encode all candidates (Frame-Voyager, FlexSelect) shorten the LLM context but spare no encoder work and can increase total latency (Sec. V).
- The strongest 2025-2026 methods combine reduction at multiple stages, and their end-to-end gains cannot be attributed to a single stage without ablations; only HoliTom reports the separating ablation.
- Field-wide efficiency evaluation is fragmented: no shared analytical protocol, near-exclusive reliance on MCQ QA benchmarks, and audio-encoder costs frequently unreported (Sec. V).
- Once LLM-side compression shrinks the token load, vision encoding becomes the limiting stage, and saved compute is often reinvested in more frames rather than realized as net speedup.
Strengths
Explicit comparison discipline
Section IV-A separates controlled shared-host comparisons (Tables II, V, VI, VII) from heterogeneous cross-paper evidence (Table IV, explicitly non-ranking), refuses to infer FLOPs or convert token budgets into FLOPs, and records accounting-boundary differences (HoliTom/HieraVid report LLM-prefill FLOPs while EarlyTom includes vision-encoder FLOPs).
Formal cost model as common backbone
Section III-B derives stage-by-stage scaling — encoder cost T·(H/P)(W/P), cross-attention (Nq+Ns)d² + Nq·Ns·d, KV memory ∝ 2·B·nlayers·L·dKV·b — and distinguishes prefill (quadratic in L) from decoding (memory-bandwidth bound), grounding the taxonomy analytically rather than in prose.
Honest negative accounting
Breadth with provenance flags
Concrete, falsifiable research agenda
Weaknesses
No original experiments
Controlled comparisons are borrowed re-runs
Self-citations in evaluation-critical positions
Audiovisual scope thinner than the title implies
Conclusions bounded to 7B-8B MCQ settings
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Preliminary Review: Why Is Video Still So Expensive? (arXiv:2609.10355)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 3 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the main conclusion of this VideoLLM efficiency survey?
That keeping about 25% of visual tokens usually preserves near-baseline accuracy on multiple-choice video QA benchmarks, while at 10% retention the choice of method matters greatly: temporal-aware methods like PruneVid and HoliTom stay close to baseline, whereas spatial-only methods like VisionZip lose more. The survey also stresses that reported speedups cannot simply be multiplied and that selector overhead must be counted.
How does this survey differ from other efficient-multimodal-LLM surveys?
It is video-specific and organizes methods by pipeline stage (frame selection and input construction, encoder computation, connector token reduction, LLM execution and KV cache), including pre-VideoLLM frame-sampling and vision-encoder antecedents, and it separates controlled same-host comparisons from cross-paper evidence. Prior surveys such as Jin et al. or Shao et al. cover broader modalities or only token compression.
Is there code or a resource accompanying the paper?
Yes, the authors maintain a public GitHub repository (momentslab/awesome-efficient-videollm) listing the surveyed methods. The survey itself runs no new experiments; supplementary material reportedly contains a fuller inventory of VideoLLM benchmarks.
Does the survey cover audio (audiovisual) efficiency?
Partly. Audio token compression and audio-guided visual selection are covered — chiefly OmniZip and DASH on Qwen2.5-Omni-7B — but the authors themselves flag audiovisual efficiency as a gap: audio encoder costs are rarely reported and modality ablations are uncommon in the literature.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt: four marked '[...sampling gap...]' regions skip parts of Sec. III-A (mid-list architecture descriptions), the end of Sec. IV-B.1(e) through IV-B.2, the start of Sec. IV-E.1, and references [170]-[174]; judgments about those specific passages rest on the visible surrounding text only.
- The supplementary material (fuller benchmark inventory) is referenced by the main text but is not part of the provided excerpt, so its contents could not be assessed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.