VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs
Haoyu Guo, Yuan Feng, Yuan Feng, Junlin Lv, Mingjun Xiao, S Kevin Zhou, Xike Xie
cs.AI · cs.CL · cs.CV · cs.MM
Quick verdict
Macro-micro inference speeds long-video MLLMs 6.13x (+7.4%) on LongVideoBench, but numeric inconsistencies and no error bars temper confidence in the gains.
Summary
VideoMM targets the token-explosion bottleneck in long-video MLLMs. Instead of auxiliary models for token selection (lightweight encoders lose semantics; small MLLMs negate speedups), it decouples selection from reasoning by resolution: Stage I runs query-aware attention scoring on a spatially downscaled macro proxy (factor k=2), partitioning frames into G=4 interleaved groups and selecting 2,048 macro tokens (top-50% frames, then top tokens); Stage II applies consensus voting across groups — if all agree, inference exits early (~70% of cases); otherwise selected regions are re-encoded at full resolution and each macro token maps to k² micro tokens. The method is training-free and evaluated with Qwen2.5-VL-7B, GLM-4.1V-9B and Qwen3-VL-8B on LongVideoBench, VideoMME and LVBench against VisionZip, VidCom and FlexSelect under a unified 8,192-token budget. Reported: 6.13× average speedup with +7.4% relative accuracy on LongVideoBench versus vanilla, and 2.73× speedup over FlexSelect at comparable accuracy; ablations cover early-exit rates, downscale factor, frame counts and task-aware routing. Limitations: internal numeric inconsistencies (text cites a 59.32 average where Table 2 shows 57.82; abstract's 7.4% versus intro's 7.0%), no variance or significance reporting, ViT re-encoding and multi-pass voting costs unaccounted in the complexity analysis, unspecified benchmarking hardware, and only three token-reduction baselines.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Fine-grained visual detail is largely unnecessary for selecting semantically relevant regions, so token selection can run on a downscaled macro proxy without sacrificing downstream accuracy (Sec 1, Sec 3.1).
- VideoMM achieves a 6.13× average speedup with a +7.4% relative accuracy gain over vanilla full-context inference on LongVideoBench across three MLLMs (Abstract; Table 1 LVB column).
- VideoMM is 2.73× faster than FlexSelect on LongVideoBench at comparable or higher accuracy (Fig 1; Tables 1–2).
- Consensus voting across frame groups acts as a reliable ambiguity router: ~70% of samples exit early at macro level, with exit rates tracking task type — 76–83% for perception, ~60% for relation reasoning (Table 3, Table 5).
Strengths
Resolution-based decoupling eliminates the auxiliary model
Unlike FlexSelect's distilled helper MLLM, VideoMM scores tokens with the target MLLM's own intermediate-layer attention on a k=2 downscaled proxy (Sec 3.2), genuinely dissolving the encoder-vs-MLLM dilemma framed in Sec 1–2 rather than trading one cost for another.
Fair, deployment-oriented evaluation design
All baselines share a unified 8,192-token budget (Sec 4.1), and Table 2 re-runs FlexSelect at 256 frames for an iso-throughput comparison — VideoMM wins accuracy there for all three models (e.g., 57.82 vs 54.62 average on Qwen2.5-VL-7B per Table 2), a stronger claim than headline speedups alone.
Unusually complete ablation suite
Negative results are reported honestly
Weaknesses
Internal numeric contradictions
No uncertainty quantification
Incomplete cost accounting and unreported hardware
Narrow baseline set and cross-table discrepancies
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
VideoMM is a method paper on efficient long-form video understanding with MLLMs. It claims (i) that the encoder-vs-MLLM token-reduction dilemma dissolves once selection is separated from reasoning by *resolution* rather than model size; (ii) a two-stage, training-free pipeline — grouped selection on a downscaled macro proxy (k=2, G=4, 2,048 macro tokens within an 8,192 budget) plus adaptive macro-micro inference governed by consensus voting with ~70% early exit; (iii) headline results of a 6.13× average speedup with +7.4% relative accuracy on LongVideoBench over vanilla, and 2.73× over FlexSelect at comparable accuracy, across Qwen2.5-VL-7B, GLM-4.1V-9B and Qwen3-VL-8B on LongVideoBench, VideoMME and LVBench. Scope is inference-time efficiency; no training is involved.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is VideoMM and how does it speed up long-video MLLMs?
VideoMM is a training-free framework for efficient long-video understanding with multimodal LLMs. It scores visual tokens using the model's own attention on spatially downscaled macro frames (2× smaller), selects a small macro-token set, then votes across frame groups: if all groups agree on an answer, inference stops early (about 70% of cases); otherwise the selected regions are re-processed at full resolution as micro tokens. The paper reports a 6.13× average speedup with a +7.4% relative accuracy gain on LongVideoBench, and 2.73× speedup over FlexSelect at comparable accuracy.
Does VideoMM require training or an auxiliary model?
No. VideoMM is presented as training-free: token scoring reuses the target MLLM's own intermediate-layer attention weights, and no additional selection model is trained, unlike FlexSelect which uses a distilled auxiliary MLLM. An optional Lite variant (Appendix D) swaps in an off-the-shelf smaller model (Qwen3-VL-4B) for selection, trading a small accuracy dip (67.54 to 66.34 on LongVideoBench with Qwen3-VL-8B) for higher throughput.
Which models and benchmarks were used to evaluate VideoMM?
Experiments cover Qwen2.5-VL-7B, GLM-4.1V-9B (thinking mode disabled) and Qwen3-VL-8B on LongVideoBench, VideoMME and LVBench, using the LMMs-Eval harness at 1 fps sampling up to 512 frames. Baselines are VisionZip and VidCom (encoder-driven) and FlexSelect (MLLM-driven), all under a unified 8,192 visual-token budget.
What are the main criticisms of the VideoMM paper?
Its numbers contradict each other in places (the text cites a 59.32 average where Table 2 shows 57.82; the abstract says 7.4% gain while the introduction says 7.0%), results carry no variance estimates or significance tests even where margins are under one point (Qwen3-VL-8B), the complexity analysis omits ViT re-encoding and multi-pass voting costs, benchmarking hardware is unspecified, and only three token-reduction baselines are compared.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–5 appear only as garbled axis-label text in the extraction; plotted frontier values (e.g., Fig 4/5) could not be verified.
- Benchmarking hardware, run counts and any hyperparameter-search details are absent from the extracted text, so absolute throughput claims cannot be assessed.
- The claimed code repository (github.com/adfh917k/VideoMM) could not be checked for availability or completeness.
- Appendix extraction is partially interleaved (e.g., Table 9 lacks the Lite row its caption and prose describe; footnote 8 text is split across pages).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.