CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Models
Pengan Chen, Kaisheng Zheng, Liang Hong, Lixia Yi, Jiyue Jiang, Jiayang Chen, Yixuan Wang, Yimin Fan, et al.
cs.CL · cs.CV
Quick verdict
Training-free KV refresh for VLM shifted prefixes reaches 97-99.5% of full-prefill at 10% budget; broad, honest eval, but baseline tuning merits scrutiny.
Summary
The paper proposes CONDUIT, a training-free inference-time policy for reusing the key-value (KV) cache of visual prefixes in vision-language models when the same images recur under a changed (shifted) text prefix. Building on norm-weighted attention analysis, it scores each cached visual token by cached-key query attention times an accessible cached-value-norm proxy, multiplies by a per-image relevance coefficient (image-mean attention relative to the cross-image mean, λ=1 by default), and refreshes only the top-ranked tokens under one global budget; with a single image the coefficient is 1, unifying single- and multi-image reuse. Across three backbones (Qwen2.5-VL-3B/7B, InternVL3-9B) and five shifted-prefix datasets (LongDocURL, MMLongBench-Doc, SlideVQA, InfoSeek, ViQuAE), a 10% refresh budget recovers 97.0-99.5% of full-prefill average accuracy and beats MPIC, CacheBlend, KVShare, and ProphetKV in all 15 dataset-backbone cells (about +1.7 points over ProphetKV on average); four single-image benchmarks confirm the K=1 limit. On a latency subset of MMLongBench-Doc, CONDUIT uses 13.5% of full-prefill FLOPs and achieves a 2.99x time-to-first-token speedup, though MPIC is faster. Limitations: static images only, a batch-size-1 serving engine without continuous batching, retained cache storage costs, a value-norm proxy resting on an indirectly validated displacement assumption, and no stated code release.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A training-free score combining cached-key query attention with an accessible cached-value-norm proxy, plus image-level relevance amplification, selects stale visual tokens for KV refresh more effectively than raw attention under shifted prefixes.
- At a 10% refresh budget, CONDUIT achieves 97.0-99.5% of the full-prefill five-dataset average across three backbones and is the highest-mean budgeted method in every one of the 15 dataset-backbone cells of Table 1.
- Single-image cache reuse is the exact K=1 limit of the same rule (coefficient c1=1), confirmed near full-prefill quality on four single-image benchmarks at r=0.05 (Table 2).
- On a context-stratified MMLongBench-Doc latency subset, CONDUIT uses 13.5% of full-prefill FLOPs and achieves a 2.99x time-to-first-token speedup, with the online scoring pass costing 26.2% of its 16k TTFT.
Strengths
Broad, seed-replicated evaluation grid
Tables 1 and 6-8 cover 3 backbones x 5 shifted-prefix datasets x 5 seeds with mean±std, plus budget sweeps at r=0.05/0.10 (Figure 3) and 4 K=1 benchmarks (Table 2); CONDUIT is the highest-mean budgeted method in all 15 dataset-backbone cells.
Honest failure analysis and limitations
Appendix D.4 audits all 23 cases where CONDUIT and full prefill diverge on MMLongBench-Doc/LongDocURL (3B, r=0.10), categorizing 15 as insufficient within-image coverage, 4 as near-tied page scores, 4 as attention-drift-consistent; the Limitations section candidly notes batch-size-1 serving, static images only, and retained storage cost.
Clean unifying formulation with verified K=1 limit
Triangulated design justification
Transparent serving-cost accounting
Weaknesses
Baseline transplant fairness is unestablished
No significance testing behind a universal per-cell win claim
Core proxy assumption validated only indirectly
No code release; custom in-house engine
Shifted-prefix construction under-specified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
CONDUIT (arXiv 2609.05821) is a training-free, inference-time policy for reusing the key-value cache of visual prefixes in vision-language models when the same images recur under a changed (shifted) text prefix. Its central claims: (1) raw attention alone is an incomplete refresh signal because high attention need not imply a large transformed write (Section 2.4, building on Kobayashi et al. 2020 and the attention-sink literature), so each cached visual token is scored by cached-key query attention times an accessible cached-value-norm proxy; (2) an empirical per-image coefficient (image-mean attention over the cross-image mean, lambda=1 by default) amplifies query-relevant images before one global top-k selection with budget floor(r*N_vis); (3) single-image reuse is the exact K=1 limit (c1=1); (4) at r=0.10 CONDUIT reaches 97.0-99.5% of the full-prefill five-dataset average on three backbones, is the highest-mean budgeted method in every dataset-backbone cell, and on a MMLongBench-Doc latency subset uses 13.5% of full-prefill FLOPs with a 2.99x TTFT speedup.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What problem does CONDUIT solve?
When a vision-language model answers a new question about images it already processed under a different text prefix, exact KV-cache reuse is invalid because the cached states are stale. CONDUIT is a training-free policy that recomputes only the roughly 10% highest-risk visual tokens (selected by query attention times cached value norm, with per-image reweighting) and reuses the rest, recovering 97.0-99.5% of full-prefill accuracy across three backbones and five document/multi-image benchmarks.
How much faster is CONDUIT than full prefill?
On a latency subset of MMLongBench-Doc with 4k-16k contexts at batch size 1, CONDUIT uses 13.5% of full-prefill FLOPs and cuts time-to-first-token by 2.99x (0.174 s vs 0.519 s averaged over three backbones). The online scoring pass accounts for 26.2% of its 16k TTFT. Note that in the same comparison MPIC was faster (5.76x), so CONDUIT's advantage is quality per unit of compute rather than raw speed.
Does CONDUIT require training or model changes?
No. It preserves the model architecture and weights and adds a single query-conditioned scoring pass at inference. All main results use a fixed image-reweighting strength of lambda=1 and all-layer score averaging with no per-benchmark tuning; with a single image the image coefficient is exactly 1, so the same rule handles single- and multi-image inputs.
What are the main limitations of the paper?
Evaluation covers static images and documents only (no video, streaming, or agentic loops); latency was measured on a compact batch-size-1 engine without continuous batching; the storage cost of retained visual K/V remains; the cached-value-norm proxy rests on a displacement assumption validated only indirectly; and no code release is stated.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2-4 are diagrams and plots; their graphical content (budget-quality curve shapes in Figure 3, qualitative page images in Figure 4, the Figure 2 pipeline diagram) cannot be verified from the text extraction beyond captions and axis labels.
- The supplied metadata lists 8 authors, but the paper's own author block lists 14 (adding Xinyuan Liu, Jiayi Li, Zhanqiu Zhang, Yiwen Guo, Yu Li, and corresponding-author marks); the paper text was treated as authoritative.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.