BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
Janghyeon Kim, Minsoo Kim, Kyuhong Shim, Jungwook Choi
cs.LG · cs.CL
Quick verdict
BeaconKV: training-free KV-cache compression for reasoning LLMs via clustered beacon queries; strong matched-budget gains, but TRT evidence is single-sample.
Summary
The paper addresses the KV-cache memory bottleneck of Large Reasoning Models (LRMs), whose long Chain-of-Thought generations grow the cache linearly. The authors observe that some decoding steps—termed Thought Revisiting Tokens (TRT)—re-attend to distant early context such as task-solving plans, and that the corresponding 'global' queries cluster into a small number of similarity groups in the pre-RoPE embedding space. Building on this, BeaconKV is proposed: a training-free compression method that maintains a compact set of 'beacon queries' selected by Continual Farthest Point Sampling, combines them with recent queries as an observation set, aligns them differentially with RoPE, and retains KV pairs via max-aggregated attention scores. On four open-source LRMs (R1-Distill-Qwen-7B/8B, Qwen3-4B/14B) and four benchmarks (AIME24, MATH-500, GPQA-Diamond, LiveCodeBench), BeaconKV generally outperforms RPC, R-KV, and SnapKV, with up to 31.7 percentage points gained over existing methods (Qwen3-14B, AIME24, 1024-token budget), up to 5.8× peak-memory reduction (77.0→13.3 GB at batch 14 with a 2K KV budget on Qwen3-4B) and over 4.3× throughput improvement while staying close to full-cache accuracy. Ablations support the (16,16) beacon/recent split and max aggregation. Main limitations: the TRT analysis rests on a single sample of one model, results lack error bars or significance tests, no code release is stated, and hyperparameter sensitivity is deferred to future work.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Certain LRM decoding steps (Thought Revisiting Tokens) re-attend distant early context, and their pre-RoPE queries cluster into a small number of similarity groups (Section 3, Figures 1-4).
- A compact set of beacon queries maintained by Continual FPS, combined with recent queries, anticipates which KV pairs future TRT will revisit without storing the full query history (Section 4).
- BeaconKV generally outperforms RPC, R-KV, and SnapKV across budgets, with gains up to 31.7 percentage points (Qwen3-14B, AIME24, 1024 budget) (Section 5.2, Figure 8).
- BeaconKV cuts peak GPU memory up to 5.8× (77.0→13.3 GB, Qwen3-4B, batch 14, 2K budget) and improves throughput over 4.3× while staying near full-cache accuracy (Section 5.4, Table 4).
- The gains are not explained by merely preserving initial context: BeaconKV beats the Initial+Recent control on all 16 model×task cells at budget 1024 (Table 3).
Strengths
Novel, mechanistically grounded observation
Section 3 identifies Thought Revisiting Tokens with concrete evidence: layer 18/head 16 on R1-Distill-Qwen-7B (AIME24 sample-0), queries 1068/1090 attending keys ~100-450 while neighbors stay local (Figures 1-2), a quantified local/global distance separation (Figure 1b; Appendix B rule: mean top-150 attention distance >200), and token-level visualization in Appendix E.
Unusually complete algorithmic specification
Algorithm 1 (cosine-FPS seeded at the minimum-average-similarity query) and Algorithm 2 (full eviction flow with pre-RoPE query buffers, Bmax_Q=32→Bmin_Q=16 compression, differential RoPE alignment, max-pooling over GQA heads) make the method reimplementable without contacting the authors.
Ablations that isolate the mechanism
Concrete, hardware-pinned efficiency results
Weaknesses
Single-trace evidentiary basis for TRT
No variance or significance reporting
No code or data release
Undiscussed aggregation anomaly and figure-only accuracy data
Efficiency overhead characterized only at 4B scale
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
BeaconKV (arXiv 2609.04971v1, ICML 2026-formatted) is a training-free KV-cache compression method for Large Reasoning Models (LRMs). Its claims: (1) LRM decoding intermittently emits Thought Revisiting Tokens (TRT) whose queries re-attend distant early context — demonstrated on layer 18/head 16 of R1-Distill-Qwen-7B on AIME24 sample-0, where queries 1068/1090 attend keys ~100-450 while neighbors stay local (Figures 1-2, Appendix E); (2) the pre-RoPE embeddings of these 'global' queries cluster into a few similarity groups (Figure 4); (3) a compact set of 'beacon queries' maintained by Continual Farthest Point Sampling (FPS), combined with 16 recent queries, suffices to anticipate which KV pairs future TRT will revisit; (4) scoring via differential RoPE (beacons rotated to current step t, recent kept at original τ) and max-aggregation over observation queries and GQA heads yields generally better accuracy than RPC, R-KV, and SnapKV, with up to +31.7 pp (Qwen3-14B, AIME24, 1024 budget), 5.8× peak-memory reduction (77.0→13.3 GB, Qwen3-4B, batch 14, 2K budget), and >4.3× throughput (Table 4).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What are Thought Revisiting Tokens (TRT) in reasoning LLMs?
Tokens generated at certain decoding steps whose attention shifts away from nearby context to distant earlier parts of the reasoning trace — for example, re-reading the task-solving plan written at the start. The BeaconKV authors show the queries of such tokens form a few similarity clusters in embedding space and use them to decide which cached key-value pairs to keep.
How much GPU memory does BeaconKV save?
In the paper's efficiency test (Qwen3-4B, 32K-token generations, batch 14, one A100 80GB), peak memory fell from 77.0 GB with a full cache to 13.3 GB with a 2K KV-token budget — a 5.8× reduction — while LiveCodeBench accuracy dropped only from 54.4% to 51.1% and throughput rose from 82.3 to 356.4 tokens/s.
Does BeaconKV require training or model modification?
No. It is training-free and leaves the architecture unchanged: it samples a small set of representative 'beacon' queries from the model's own past pre-RoPE query embeddings using farthest point sampling, and uses their attention scores (max-aggregated together with recent queries) to choose which KV pairs to evict.
Which models and benchmarks was BeaconKV evaluated on?
Four open-source reasoning models — R1-Distill-Qwen-7B, R1-Distill-Llama-8B, Qwen3-4B, and Qwen3-14B — on AIME24, MATH-500, GPQA-Diamond, and LiveCodeBench, compared against RPC, R-KV, and SnapKV under matched KV budgets ranging from 128 to 4096 tokens.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-9 appear in the extraction only as captions and axis labels; exact per-point values of the Figure 8 accuracy curves are unavailable outside prose and Tables 1-6, so per-cell baseline numbers rely on the authors' text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.