Jacap: Robust KV Cache Eviction via Jacobian-Based Nonlinear Information Capacity Preservation
Jiaming Yang, Chenwei Tang, Liangli Zhen, Chenyang Zhang, Jiancheng Lv
cs.CL
Quick verdict
JACAP extends the authors' linear CAPKV with a softmax-Jacobian capacity objective for KV eviction; theory is clean but the deployed score drops the competition terms, and gains are untested statistically.
Summary
The paper proposes JACAP, a KV-cache eviction method for long-context LLM inference. It models retained attention as a noisy nonlinear channel, linearizes the softmax attention map around a query center via its Jacobian (Lemma 1), and derives a log-determinant information-capacity objective (Theorem 1) that adds softmax sensitivity to the authors' prior linear-Gaussian framework (CAPKV). For efficiency, the softmax competition matrix and query-response covariance are both diagonalized, yielding a per-token score combining attention-derived sensitivity, a quadratic query term, and a leverage-score diversity term. Experiments cover LongBench (Qwen3-8B/14B, Llama3.1-8B; ratios 0.25-0.9), NIAH, and decoding-stage eviction on AIME25. At 0.9 compression JACAP leads all baselines on all three models (e.g., 40.91 vs 36.57 CAPKV on Qwen3-8B); at 0.25 it is consistently second to CAPKV. Runtime is comparable to baselines. Limitations: no statistical testing anywhere; AIME25 gaps equal one problem out of a small set; the diagonal approximations discard the pairwise competition the nonlinear framework was motivated by, with no ablation of these choices; key hyperparameters (query-statistics estimation, eviction cadence) are underspecified; no code release.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Modeling retained attention as a nonlinear channel and linearizing via the softmax Jacobian yields a capacity objective (Theorem 1) that adds softmax sensitivity and competition to the linear-Gaussian CAPKV objective.
- The practical JACAP score (Algorithm 1: wᵢ = ᾱᵢ²(1−ᾱᵢ)²κᵢ/d_k times leverage uᵢᵀA⁻¹uᵢ) preserves the essential structure of the Jacobian capacity at heuristic-method cost.
- JACAP outperforms CAPKV, SNAPKV, KEYDIFF, EA, and KNorm on LongBench, NIAH, and AIME25, with the largest gains at high compression (≈5 points at C.R.=0.9 on LongBench averages).
- The runtime overhead of the capacity-matrix computation is negligible in practice for typical head dimensions (d_h=128), per Appendix C.1.
Strengths
Clear theoretical delta over prior work
Lemma 1/Theorem 1 derive the softmax Jacobian SC = Diag(α*) − α*α*ᵀ into the capacity objective, with linear capacity (Eq. 2) recovered as a special case; the Appendix A.1 derivation is complete and correct.
Honest saturation analysis
Appendix A.2 explicitly notes tokens with attention near 0 or 1 have near-zero local sensitivity Sᵢᵢ, contrasting with pure attention-based scoring — a genuinely useful insight versus H2O/SNAPKV-style heuristics.
Strong high-compression results
Runtime empirically validated
Weaknesses
Deployed score discards the paper's motivating structure
No statistical reporting at all
Underdetermined query-statistics estimation
Mixed low-compression picture unexamined
No code release; reference hygiene issues
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
JACAP — Preliminary Review
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Jacap (Jacobian-Capacity Aware eviction) in one sentence?
It is a KV-cache eviction method for long-context LLM inference that scores each cached token by a softmax-sensitivity weight (peaking at moderate attention, near zero for saturated tokens) times a diversity term, derived from a Jacobian-based information capacity objective.
How does Jacap differ from CapKV?
CapKV (the same authors' earlier method) scores tokens under a linear-Gaussian attention model; Jacap linearizes the actual softmax attention around a query center and adds a sensitivity factor α²(1−α)², though in the final algorithm both the competition matrix and query-response covariance are diagonalized, so the practical difference is the per-token weighting plus the same leverage-score selection.
When does Jacap help most?
According to the paper, under aggressive cache compression: on LongBench at compression ratio 0.9 it averages roughly 40.9/44.6/36.0 on Qwen3-8B/Qwen3-14B/Llama3.1-8B, about 4–5 points above the best baseline, while at the mild ratio 0.25 it is slightly behind CapKV.
Is code for Jacap available?
The preprint provides no code repository or release statement; reproduction requires reimplementing Algorithm 1, and details such as how the query statistics μQ and ΛQ are estimated during decoding are not fully specified.
Why does the paper claim high-attention tokens can be unimportant?
Because the softmax Jacobian's diagonal is α(1−α): a token whose attention weight is near 1 (dominant) or near 0 (ignored) has almost no local sensitivity to query changes, so its marginal information contribution is argued to be lower than that of moderately-attended, responsive tokens.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- NIAH results (Figs. 3/6) are heatmaps whose values are not extractable as text; assessment of them relies on the prose claims only.
- The runtime figure (Fig. 5) numeric labels are present but extraction interleaves series, so per-method values were not individually cited.
- A few Table 1 cells show minor OCR mangling (e.g., '27⟨20' in Llama3.1-8B KNorm C.R.=0.75) without affecting read conclusions.
- References [19] and [20] appear as separate entries but cite the same arXiv report (2407.18003).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.