ValueDiff: Value-Geometric KV Cache Eviction for Sink-Suppressed LLMs
Junyoung Park, Jungwook Choi, Mingu Lee
cs.LG · cs.AI
Quick verdict
ValueDiff evicts KV tokens by value-vector distance from the cache mean; strong gains on sink-suppressed models, but no significance testing on main results.
Summary
The paper targets KV cache eviction for LLMs whose architectures (QK-normalization, gated attention, learned sinks, logit softcapping) weaken attention sinks, undermining key-side eviction signals. Across 12 models, the authors observe that weaker sinks co-occur with higher value-to-key dispersion (σV/σK; Spearman 0.76). They propose ValueDiff, which scores each cached token by the L2 distance of its value vector from the cache mean and evicts the lowest-scoring tokens at every 128-token prefill block and decoding step. The score is derived as the minimal-disturbance eviction under a uniform/max-entropy (Dirichlet) assumption on future attention. On RULER (8k context, 2k budget), ValueDiff retains 88-99% of dense accuracy and is best on 6 of 7 sink-suppressed models; on LongBench (4k budget) it averages ~92% retention versus ~83% for FastCAOTE, the strongest prior baseline; on MATH-500 it leads non-dense methods on sink-suppressed models at the 25% budget, by ~19-24 points on gated-attention Qwen3.5 models. Limitations: no significance testing or error bars on the main tables; MATH-500 uses only the first 100 problems; Qwen3.5 is evaluated under a different greedy no-think protocol; the sink-geometry correlation is observational and confounded by architecture differences; no code release is stated.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Weaker attention sinks co-occur with higher value-vector dispersion relative to keys (σV/σK) across 12 models spanning four suppression mechanisms (Spearman 0.76).
- The score s_j = ‖v_j − v̄‖ is the minimal-disturbance KV eviction under uniform/max-entropy future attention, exactly under Dirichlet(1,...,1).
- ValueDiff is the best or tied-best non-dense eviction on sink-suppressed models: 6 of 7 on RULER at 2k budget (88-99% dense retention), ~92% vs ~83% average on LongBench at 4k, and up to ~20 points ahead at MATH-500's 25% budget.
- Key geometry leads only where the BOS anchor is intact (bos-cosim < 0 and σV/σK < 1, Quadrant III of Section 4.4).
Strengths
Novel, measurable motivation linking architecture trend to geometry
Figure 1 and Tables 4-6 show sink rate vs σV/σK across 12 models (Spearman 0.76, Pearson 0.84), and three independent metrics — weight spectral ratio, activation σ1 ratio, and L2 dispersion — preserve the same architectural ordering (Table 6), strengthening the observation beyond a single measurement.
Correct and cleanly presented theory
The Dirichlet(1,...,1) derivation in Appendix A is verifiable: Beta moment E[αj²]=2/(n(n+1)), Dirichlet independence factorization, and reduction to 2/(n(n−1))·‖vj−v̄‖²+C(n) all check out; Appendix B bounds deviation from the uniform limit via δ(α).
Unusually well-matched baseline design
Candid negative-case analysis
Two-axis predictive picture
Weaknesses
No statistical rigor on headline tables
MATH-500 subset size and mixed protocols
Observational core claim with architectural confounds
Modest evaluation scale and coverage gaps
No code release stated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ValueDiff (Qualcomm AI Research; arXiv 2609.23314) targets KV cache eviction for "sink-suppressed" LLMs — models with QK-normalization (Qwen3-4B, Gemma3-4B), gated attention (Qwen3.5-4B/9B), learned sinks (GPT-OSS-20B), or logit softcapping (Gemma2-2B/9B) — where classic sink-preserving and attention-score eviction lose their anchor (sink rates near 0.01 on Qwen3.5 vs 0.80+ on Llama, Table 4). Three claims: (1) weaker sinks co-occur with value-dominant geometry (σV/σK up to 1.67 on GPT-OSS-20B vs 0.33 on Llama 3.1-8B; Fig 1, Tables 4-6); (2) the score s_j = ‖v_j − v̄‖ is the minimal-disturbance eviction under uniform/max-entropy future attention (Prop 3.1, Appendix A); (3) empirically, ValueDiff is the best non-dense method on sink-suppressed models across RULER, LongBench, and MATH-500.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ValueDiff in one sentence?
A training-free KV cache eviction method that keeps tokens whose value vectors are farthest (in L2 distance) from the mean value vector of the current cache, evicting the closest-to-mean tokens first at every prefill block boundary and every decoding step.
Why do existing KV cache eviction methods struggle on newer LLMs?
Most methods rely on attention sinks or key-side signals. Architectures with QK-normalization (Qwen3, Gemma3), gated attention (Qwen3.5), learned sinks (GPT-OSS-20B), or logit softcapping (Gemma2) have much weaker sinks — near-zero sink rates on Qwen3.5 and GPT-OSS-20B per the paper — so attention scores become unstable proxies for future token importance.
How much better is ValueDiff than prior methods?
On the paper's numbers: on RULER at 8k context with a 2k budget it retains 88-99% of dense accuracy and is best on 6 of 7 sink-suppressed models; on LongBench at a 4k budget it averages ~92% retention versus ~83% for FastCAOTE, the strongest prior baseline; on MATH-500 at a 25% budget it leads by roughly 19-24 points on the gated-attention Qwen3.5 models.
Does ValueDiff require training or calibration data?
No training is involved. The eviction rule itself is query-invariant and computed on the fly from the cache; the paper's diagnostic geometry statistics used 20 calibration prompts, but the score (L2 distance from the cache mean) needs no learned components.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 and 8 are extracted as scattered text; visual trends (e.g., Figure 2 quadrant assignments) were assessed via the prose and captions, not the plots.
- Table cell alignment in the extracted text could in principle misalign columns; headline numbers were cross-checked against the prose claims where possible (6-of-7, 92% vs 83%, 19-24 points).
- No access to source code or generation scripts; implementation-dependent claims (TTFT) judged from the text alone.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.