CacheBridge: Efficient Cross-Model KV Cache Transfer
Xingyu Qu, Siyuan Lu, Zhiyu Chen, Sheng Wang, Tao Lin
cs.AI
Quick verdict
CacheBridge refines closed-form cross-model KV-cache transfer with head-local support and attention-aligned fitting; solid evidence, same-family pairs only.
Summary
CacheBridge targets the problem that KV caches are model-specific, so a receiving LLM otherwise re-prefills a shared prefix. Building on Full-Head Mapping (Heo et al., 2026), a training-free closed-form affine mapper from source to target caches, the paper identifies four weaknesses: model-sensitive accuracy, a KV-fit objective misaligned with receiver attention, handoff cost growing with layer support, and fragmented top-k construction. CacheBridge co-designs three fixes while keeping the deployed artifact affine: Head-Local restricts each target KV head to its architecture-indexed source head (8x fewer coefficients), Attn-Repair weights ridge residuals by causal attention sensitivity with an effective-sample-size floor, and Fused-Fit builds weighted sufficient statistics with a fused GPU kernel. On Ministral 3 3B-to-14B and 8B-to-14B, HellaSwag rises from 52.2%/44.4% to 72.6%/76.0% (mean retention 88.23%/97.57% vs 65.89%/59.43%); Qwen3 14B-to-32B retention stays at 99.83%. On Qwen3, mapper storage falls from 4.296 GB to 0.538 GB, application speeds up to 3.0x, and 500-sequence construction drops from 92.63 s to a median 8.63 s. An attribution study isolates each component. Limitations: same-family GQA pairs with matched KV-head counts and identity head assignments only, no variance or significance reporting, no code release, and latency measured for mapper application only.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Restricting each target KV head to one architecture-indexed source head (Head-Local) repairs both Ministral 3 failure directions while cutting coefficient count 8×.
- Attention-aligned calibration (Attn-Repair) improves continuation quality (retention +1.25 points, NLL@4K 2.446→2.350 on Qwen3) without improving coordinate-level R², showing KV fit is an insufficient proxy.
- CacheBridge preserves 99.83% mean target retention on Qwen3 14B→32B and matches full-head mapping's retention with one tenth of the calibration data on a frozen sweep.
- A fused GPU construction kernel builds the 500-sequence mapper in a median 8.63 s versus 92.63 s (10.7×) on four H800s without changing the fitted artifact or online affine interface.
Strengths
Clean attribution design isolates each mechanism
Section 4.3 / Table 2 runs capacity-matched controls on Qwen3 14B→32B: a block-PCA control at Head-Local's coefficient budget loses 13.97 points, seven cyclic head-shift assignments at matched width lose a mean 15.50 points (Fig 5), and Attn-Repair changes only calibration weights (+1.25 retention, NLL@4K 2.446→2.350) while K/V R² barely move (0.678/0.655→0.672/0.654) — directly supporting the claim that aligned support, not capacity, drives the gain.
Repairs real baseline failures, not just marginal wins
Full-Head Mapping collapses on the two Ministral 3 directions (52.2% and 44.4% HellaSwag); CacheBridge restores 72.6% and 76.0%, with four-task mean retention rising 65.89%→88.23% and 59.43%→97.57% (Table 1) — a targeted fix rather than noise-level improvement.
Efficiency claims are specific and consistent
Honest, well-scoped limitations section
Weaknesses
Headline mechanism never exercised beyond the identity map
No variance, seeds, or significance anywhere
Unexplained anomaly: ARC-Challenge retention above 100%
Efficiency wins are mapper-local, not end-to-end
Ablations confined to one direction; learned baselines not run
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
CacheBridge (arXiv 2609.00891) addresses cross-model KV-cache reuse: in multi-LLM systems the receiver normally re-prefills a shared prefix because caches are model-specific. The paper builds directly on Full-Head Mapping (Heo et al., 2026, arXiv 2608.03893), a training-free closed-form ridge mapper from source to target caches, and claims four of its limitations (O1–O4, §1, Fig 1): model-sensitive accuracy (52.2%/44.4% HellaSwag on the two Ministral 3 directions versus 79.2% on Qwen3 14B→32B), a query-free KV-fit objective misaligned with continuation quality, handoff cost growing with selected-layer count (approaching target re-prefill at k≈16 and exceeding it at k≥20), and fragmented top-k construction. Three components follow, all preserving the affine deployment interface: Head-Local (one architecture-indexed source head per target head, 8× fewer coefficients), Attn-Repair (ridge sample weights from first-order causal attention sensitivity with a Kish effective-sample-size floor, Eqs. 17–19), and Fused-Fit (a fused GPU kernel accumulating weighted sufficient statistics without materializing full observation tensors, Eqs. 20–22).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is CacheBridge and what problem does it solve?
CacheBridge is a training-free method that lets one LLM reuse another LLM's KV cache (the stored attention states of a shared prompt prefix) instead of re-processing the entire prefix. It fits a closed-form linear (affine) mapping from the sender model's cache to the receiver model's cache, so the receiver can resume decoding immediately. Compared with prior full-head linear mapping, it links each target attention head to one matched source head, weights calibration errors by how much they disturb the receiver's attention, and builds the mapper with a faster fused GPU routine.
How much faster and smaller is CacheBridge than the baseline?
On Qwen3 14B→32B, the paper reports the serialized mapper shrinking from 4.296 GB to 0.538 GB (8×), mapper application up to 3.0× faster (e.g., 21.66 ms vs 65.12 ms for a 1,024-token prefix), and construction on 500 calibration sequences dropping from 92.63 s to a median 8.63 s (10.7×) on four NVIDIA H800 GPUs. These numbers cover the mapper itself, not full request latency.
Does CacheBridge work between different model families?
Not demonstrated. All three evaluated transfer directions connect models within the same family (Ministral 3 3B/8B→14B and Qwen3 14B→32B), and every model uses grouped-query attention with exactly eight KV heads, so the head-assignment rule degenerates to the identity map. The authors list cross-family transfer and mismatched head counts as open limitations.
What accuracy does CacheBridge achieve?
On the two Ministral 3 directions where the full-head baseline collapses (HellaSwag 52.2% and 44.4%), CacheBridge reaches 72.6% and 76.0%; mean retention across four benchmarks rises to 88.23% and 97.57% from 65.89% and 59.43%. On Qwen3 14B→32B both methods are near-native (99.83% vs 99.72% mean retention), with CacheBridge slightly better on most tasks.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text ends after the reference list with no appendix; fused-kernel implementation details and any supplementary material are genuinely absent from the extraction.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.