KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints
cs.CL
Quick verdict
Benchmark of KV-cache reuse across prompt contexts and model checkpoints; rigorous floor/ceiling PGR scoring, credible class findings, single-model rankings.
Summary
KVShareArena is a benchmark for reusing KV caches when the prompt context changes and/or when the cache was produced by a different checkpoint of the same architecture. Under a fixed receiver it crosses these two factors across two tracks — Retrieved Evidence (three LongBench v1 QA subsets plus a FRAMES variant over a checksum-pinned 2026 Wikipedia snapshot) and Agent Reports (question-blind specialist reports) — with 100 frozen samples per subset. Every method is scored by PGR, the fraction of the gap it recovers between a no-cache floor and a full-recomputation ceiling, alongside measured compute, memory, and same-backend TTFT; the one-time cache build is reported separately. Findings: free position alignment suffices while source interaction is weak; when questions need several mutually blind sources, only selective recomputation (CacheBlend at a measured 17%, LegoLink under 0.5%), calibration (APE), and the trained adapter KVPacket recover .36-.46 of the gap on multi-hop QA, while naive assembly sits below the floor. Compression never significantly beats free alignment and falls well below it on fresh reports. Cross-checkpoint reuse barely moves training-free methods but breaks producer-fitted trained repairs. Patterns replicate on Llama-3.1-8B and a 4B sibling. Limitations: per-row rankings rest on one 8B model; N=100 yields ~.10 resolution; latency covers only eight same-backend rows; released links are placeholders.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Free position re-alignment is sufficient until a task requires several mutually blind sources jointly; only there do paid repairs earn their cost.
- On multi-hop QA, paid repairs recover .36-.46 of the no-cache-to-recompute gap while naively assembled caches score below the no-cache floor.
- KV compression that is mild in its birth context degrades severely after misaligned assembly and never significantly beats free alignment.
- Cross-checkpoint reuse leaves training-free methods largely unchanged but breaks repairs trained on one producer's KV distribution, failing silently with fluent wrong-entity answers.
- No prior evaluation crosses the context and checkpoint factors with floor/ceiling scoring and shared-backend cost accounting (Table 1).
Strengths
Genuinely novel two-factor evaluation design
Table 1 documents that no prior work crosses context change with checkpoint change; the 2x2 under a fixed receiver (Section 2.2, Table 7) cleanly separates geometry from weight mismatch — on HotpotQA direct reuse, same-weights cross-context drops .559 to .394 while foreign-weights same-context only drops to .484/.527.
Normalized scoring with pre-registered, enforced admission
PGR between a frozen no-cache floor and same-payload ceiling makes failures legible (naive assembly PGR -.824 on Agent Reports, Table 3). Admission criteria (C1 ceiling-floor >= .25, C2 residual >= .25, C3 paired significance) were committed before runs and negative admissions are published: a memory-track candidate failed C3 (repair events 6/60), a summarization candidate failed C2 on both tracks, two candidates died at the anchor stage.
Unusual measurement integrity for a systems benchmark
Class-level findings supported by dose-response controls
Honest cost accounting with measured, not nominal, quantities
Weaknesses
Per-row rankings rest on a single 8B model
Significance calls are instrument-dependent
Small samples put several conclusions near the resolution floor
Sparse and partly self-reported cost coverage
Agent Reports track depends on an author-constructed pipeline with acknowledged style confounds
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
KVShareArena — Preliminary Review
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is KVShareArena?
KVShareArena is an open benchmark (pip package plus public leaderboard) that measures how well methods repair KV caches reused outside their original context — for example RAG chunks assembled in a new order, or agent reports handed to a coordinator — and caches written by a different checkpoint of the same model family. Methods are scored by PGR: the fraction of the quality gap they recover between answering with no cache and fully recomputing the payload, alongside compute, memory, and per-request latency costs on a shared backend.
What are KVShareArena's main findings?
Free position re-alignment is enough until a question must jointly use several mutually blind cached sources. There, only methods that recompute part of the cache or use trained components (selective recomputation such as CacheBlend and LegoLink, attention calibration such as APE, and the trained adapter KVPacket) recover roughly .36-.46 of the gap on multi-hop QA; naively assembled caches score below the no-cache floor (PGR -.82 on agent reports). Compression never significantly beats free alignment and is significantly worse on fresh reports. Changing the producer checkpoint barely affects training-free repairs but breaks repairs trained on one producer's caches, failing silently with fluent but wrong answers.
Which models and datasets does KVShareArena use?
The main board uses Qwen3-8B (identified in Figure 5's caption) as both cache producer and receiver; Llama-3.1-8B-Instruct and a 4B sibling serve as confirmation boards. Retrieved Evidence draws on three LongBench v1 QA subsets (scientific-paper, single-document, multi-hop) plus a FRAMES variant over a checksum-pinned 2026 Wikipedia snapshot; the Agent Reports track uses question-blind specialist reports. Each subset has 100 frozen samples with SHA256 manifests and a 29K-token input cap.
Can I use KVShareArena's results to choose a method for my RAG server?
Cautiously, yes at the class level. The guidance is consistent across three model boards: start with free position re-alignment; pay for recompute-based repair (e.g., CacheBlend at ~17% recompute or LegoLink at under 0.5%) only when queries must combine several cached sources at once; avoid plain cache compression under reassembly. Exact per-method rankings come from one 8B model with N=100 samples per subset (~.10 resolution), so treat individual row orderings as provisional.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 3, 5, and 6 are flattened into garbled axis-label text in the extraction; all per-method numbers cited in this review were read from the intact Tables 2-5, 7-9 and appendix prose.
- All artifact links appear as '[repo link]' / '[dataset link]' / '[leaderboard link]' placeholders throughout, consistent with double-blind review, so the claimed releases could not be inspected from the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.