Shared Global KV with Layer-Specific Local History
cs.LG
Quick verdict
Reports 1.4% PPL gain from local history beside shared global KV at 126M, rigorously controlled; tiny scale and n=2 design limits transfer.
Summary
The paper proposes a decoder-only transformer in which eight lower layers produce a single shared global key-value (KV) cache while each of eight upper layers additionally keeps a bounded 128-token local history read through one joint softmax. The central question is what this local memory should retain. In an eight-seed factorial at 126M parameters and 2K context trained on 1B FineWeb-Edu tokens, replacing a current-token local branch with history lowers held-out perplexity by 1.37% on development and 1.41% on reserved test text; capacity, entry-multiplicity (repeated-current) and compute-matched controls support the benefit. A two-seed study shows history remains useful when adjacent layers share local inputs with independent projections, which also shortens exact cache-construction dependencies. Against GQA2 and GQA4-CLA2 under equal bounded learning-rate searches, the design achieves better likelihood but uses 19.63% more cache and up to 32.98% more request time at the long-decode batch-one workload; the ordering persists after equal-token 8K adaptation, with a ~2.8% short-context cost. Downstream tasks are mixed: PIQA favors the alternatives and ARC-Easy favors GQA in one comparison, while the external-book (PG-19) history effect remains statistically unresolved. A proven suffix schedule reduces upper-layer prefill work while preserving the complete cache in exact arithmetic. Main limitations: small scale, two-seed architecture comparisons, and FP32-only cost measurements.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- At 126M/2K, replacing a current-token local branch with 128-entry local history lowers held-out perplexity ~1.4% (1.37% dev, 1.41% reserved), with 8/8 paired seeds favoring history.
- The history benefit exceeds entry multiplicity and persists under matched training compute and exact parameter-count controls.
- Adjacent-pair sharing of local inputs (with independent projections) preserves the history benefit and shortens exact cache-construction dependencies.
- Against GQA2 and GQA4-CLA2, the design yields better same-source likelihood at +19.63% cache and up to +32.98% long-request latency, and the ordering persists after equal-token 8K adaptation.
- A proven suffix schedule (Proposition 1) constructs the complete cache exactly from shorter per-layer suffixes, reducing upper-layer prefill work.
Strengths
Prespecified eight-seed factorial with family-adjusted inference
Endpoints, comparisons and interval levels (98.75% family-of-four, 97.5% family-of-two Bonferroni) fixed before scoring; all 32 endpoints published in Table 5 with no seed dropped; BCa sensitivity checks agree with t-intervals; 8/8 paired seeds favor history (−1.37% dev, −1.41% reserved PPL).
Controls that genuinely isolate the mechanism
The repeated-current control fixes entry multiplicity via the exact log m_t score offset (Eq. 3); a rank-256 pointwise adapter matches parameter count exactly (126,248,448); a compute-matched arm matches counted matrix-product FLOPs to <0.001% — history still wins (−1.02% reserved, −1.76% books, §4.3).
Honest cost accounting without speedup overclaiming
Exact-arithmetic suffix schedule with proof and numerical qualification
Unusual disclosure quality
Weaknesses
Architecture conclusions rest on two seeds
Likelihood-task dissociation unresolved
Small scale limits transfer
Serving economics measured in FP32 only
Small effect against a measurable premium
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper studies KV-cache allocation in decoder-only transformers: eight lower provider layers build a shared global KV bank, while each of eight upper layers additionally keeps a bounded 128-token local bank from its own features, both read in a single softmax (Eqs. 1–2, Figure 1). The design deliberately separates two interventions usually conflated: what local memory retains (history vs current token) and what input forms it (per-layer vs adjacent-pair). The claims are: (1) historical content beats a current-token branch by ~1.4% PPL at 126M/2K (§4.2); (2) the benefit survives capacity, entry-multiplicity and compute-matched controls (§4.3); (3) it persists when adjacent layers share local inputs with independent projections (§4.4); (4) the whole design gives better likelihood than GQA2 and GQA4-CLA2 at measured costs of +19.63% cache and +32.98% request time on the long-decode batch-one workload (§4.5, §5.1); (5) Proposition 1 gives a provably exact suffix prefill schedule (§5.2).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'shared global KV with layer-specific local history' mean?
It is a transformer architecture where a stack of lower layers builds one shared global KV cache that all upper layers reuse, while each upper layer additionally keeps a small sliding-window KV (the last 128 tokens) formed from its own features; each upper layer's query attends to both sources through a single softmax.
How much does the local history improve over alternatives?
At 126M parameters and 2K context, keeping history instead of only the current token lowers perplexity by about 1.4% (1.37% development, 1.41% reserved test, eight paired seeds). Against GQA-style KV-sharing baselines it gives better likelihood, but uses 19.63% more cache and up to 32.98% more request time on long-decode batch-one workloads; zero-shot task accuracy shows no consistent advantage.
What is Proposition 1 in this paper?
It is a proof that, because upper-layer dependencies are bounded by the 128-token window, the complete KV cache and prompt-end output can be reconstructed exactly from shorter per-layer suffixes — e.g., 4,580 KV-input and 3,564 query/output positions instead of 8,136 positions for a uniform suffix at N≥1017 — which cuts upper-layer prefill work by roughly 1.8x in the tested setting.
Is code or model weight access available for this paper?
The paper references a companion research archive containing numerical records, recomputation scripts, seed-identity mappings and portable training/scoring entry points; pretrained weights and raw source text are excluded, and the authors note full retraining has not been rerun from the portable adapter.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Text extraction omitted parts of the §4.4–4.5 narrative and tails of some appendix tables (e.g., Table 18, Table 27); judgments rely on the preserved numbers and captions.
- Figures are not analyzable from the extracted text; only captions and in-text numbers were used.
- The AI-use statement discloses Codex assistance; the companion archive could not be independently verified offline.
- arXiv listing (2609.28006, cs.LG) and some cited works (e.g., DeepSeek-V4.1-Flash, 2026 references) postdate typical verification; citation accuracy was taken at face value.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.