Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Vishesh Tripathi, Abhay Kumar, Ramsha Khan
cs.AR · cs.LG
Quick verdict
Proposes GVA, caching grouped values with query-absorbed key reconstruction (~45-47% cache cut vs GQA); near-GQA accuracy at 350M, but no systems results yet.
Summary
Grouped Value Attention (GVA) is a KV-cache design for Transformer decoding. It keeps GQA's grouping on the value side but drops the key stream entirely: only G grouped value sequences are cached, and each query head's content key is reconstructed with a learned per-head linear map K = VM. At inference the map is absorbed into the query, so content keys are never materialized or stored; positional information is carried by a small shared decoupled-RoPE channel cached separately. For the configurations studied, the intended persistent cache shrinks by roughly 45-47% versus matched GQA — a representation-level scalar count, not a measured serving-memory figure. Training 350M-parameter models from scratch on 30B FineWeb-Edu tokens, the dr=16 variant averages 44.35 over five zero-shot tasks with three seeds per configuration, versus 44.36 for GQA and 43.88 for MLA; an earlier shared-KV design (K=V) fails to recover GQA loss. Main limitations: no latency, throughput, or peak-memory measurements exist (the custom decoding kernels are unreported), only one model scale and one dataset are tested, no seed-dispersion statistics back the 'within seed variation' parity claim, and the added per-head map parameters are not accounted against GQA. An open-source release is planned but not yet available.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- GVA caches G grouped value streams plus one shared positional key and reconstructs per-head content keys via a learned linear map that is exactly absorbed into the query at decode.
- For the configurations studied, this cuts intended persistent cache scalars by roughly 45-47% versus matched GQA (representation-level counts, not measured memory).
- At 350M parameters on 30B FineWeb-Edu tokens, GVA + decoupled RoPE (dr=16) reaches 44.35 average accuracy over five zero-shot tasks versus 44.36 for GQA and 43.88 for MLA, averaged over three seeds each.
- Sharing one vector as both key and value (K=V) halves the cache but fails to recover GQA training loss, motivating the learned reconstruction.
- Custom decoding kernels implementing the absorbed path have been developed, but no end-to-end inference gains are established in this paper.
Strengths
Exact, verifiable absorption algebra
Section 3.2 (Eq. 7) shows q-tilde = qMᵀ reduces content scoring to a single inner product with the cached value; the identity is exact, and the argument that standard RoPE breaks absorption (relative rotation depends on the query-key position pair, sitting between qt and Mᵀ) is correctly derived.
Honest scoping of what is and is not measured
Sections 1, 4.2, 6 and 7 repeatedly state that cache figures are representation-level counts, not measured serving memory, and that no latency/throughput/peak-memory numbers exist — unusually candid framing for an efficiency paper.
Negative result included and used
Principled initialization with derivation and ablation
Internally consistent cache arithmetic and tables
Weaknesses
No systems validation for a systems claim
Thin statistical reporting
Single scale, single dataset, short context
Unquantified added cost
Under-specified training recipe
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper proposes Grouped Value Attention (GVA), a KV-cache representation for decoder-only Transformers. GVA keeps GQA's grouping on values, caches no content keys, and reconstructs each query head's content key as Kh = Vg(h)Mh (Eqs. 1, 6); since Mh is fixed at inference it is absorbed into the query (Eq. 7), so the intended decode path stores only G grouped value streams plus one shared positional key of width dr (Algorithm 1, Eqs. 12-13). Claims: (1) ~45-47% fewer persistent cache scalars than matched GQA; (2) near-parity quality — at 350M parameters on 30B FineWeb-Edu tokens, dr=16 averages 44.35 over five zero-shot tasks vs 44.36 (GQA) and 43.88 (MLA), three seeds each (Table 2); (3) shared KV (K=V) fails, motivating learned reconstruction (Figure 3); (4) custom decode kernels exist but are unreported. Note: the provided abstract gives 44.18 for dr=16 while the v2 full text and Table 2 give 44.35; this review uses the full-text figures.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Grouped Value Attention (GVA)?
GVA is a KV-cache design for Transformer decoding. Instead of caching both keys and values per group as GQA does, it caches only grouped value vectors and reconstructs each attention head's key with a small learned linear map. Because that map is fixed at inference, it can be folded into the query, so keys never need to be stored; a small shared positional (RoPE) vector is cached separately to retain position.
How much KV cache memory does GVA save?
For the configurations studied, roughly 45-47% fewer cached scalars than matched GQA — about half comes from dropping the key stream, minus a small shared positional key. These are representation-level counts, not measured serving memory.
Does GVA match GQA in accuracy?
At 350M parameters trained on 30B FineWeb-Edu tokens, the 16-dimensional positional variant averaged 44.35 across five zero-shot tasks over three seeds, versus 44.36 for GQA and 43.88 for MLA. The gap is tiny but not statistically established, since no seed dispersion is reported.
Is GVA faster than GQA or MLA at inference?
Not yet demonstrated. The authors built custom decoding kernels but report no latency, throughput, or peak-memory measurements; an open-source release is planned.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-10 are image content; curve-level claims (e.g., loss trajectories in Figures 2-3) rest on captions and in-text description only.
- The provided metadata abstract reports 44.18 for the dr=16 variant while the extracted full text (v2) and Table 2 report 44.35; this review uses the full-text numbers.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.