HySparse2: Hybrid Sparse Attention with Two-Level KV Sharing
Jianyu Wei, Yizhao Gao, Qihao Zhang, Shimao Chen, Zhengju Tang, Yu Cheng, Shengjie Zhou, Zihan Jiang, et al.
cs.CL
Quick verdict
Proposes two-level KV sharing sparse attention that cuts prefill and KV cache with better retrieval; solid, though MQA switch confounds comparisons.
Summary
HySparse2 is a hybrid sparse-attention architecture from Xiaomi's LLM-Core team targeting long-horizon, multi-turn agent inference. It organizes a 49-layer 80B-A3B MoE backbone into a self-decoder (hybrid full/sliding-window attention) and a cross-decoder (hybrid full/sparse attention), with two-level KV sharing: outer-level KV Bridging projects self-decoder full-attention hidden states into cross-decoder full-attention KV caches via layer-specific projections, while inner-level KV Reuse lets sparse layers share the full layer's KV cache and selection indices. Two refinements over HySparse — token-level (rather than block-level) sparse selection and a forced 128-token recent window replacing a separate SWA branch — allow prefill to exit after the self-decoder. Trained on ~500B tokens plus ~100B light post-training, HySparse2 matches baselines on general benchmarks (MMLU 63.92) and improves long-context retrieval: post-training mean MRCR-v2 +11.30 and RULER-v2 +19.81 points over HySparse, with 2.92×/5.02× prefill-FLOPs reductions at 1M tokens versus HySparse/Hybrid SWA and a 2.69 GB KV cache versus 6.72/12.09 GB. Main limitations: comparisons conflate the attention design with a GQA-to-MQA head-configuration change; key benchmarks (AgentPPL, Repo Code PPL) are internal; no code or model release; no variance estimates or wall-clock throughput; and end-to-end agent task success is not evaluated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Two-level KV sharing lets all cross-decoder KV caches be constructed from self-decoder hidden states, so prefill can exit after the self-decoder (25 of 49 layers on the prefill node).
- Token-level sparse selection improves long-context retrieval over block-level selection at the same attention budget (Table 3: RULER-v2 +6.57, 2-needle MRCR-v2 +8.14, GraphWalks +5.55).
- Forcing a 128-token recent window into sparse selection is competitive enough with a separate gated SWA branch (Table 4) to enable fully skipped cross-decoder prefill.
- After light post-training on 80B-A3B models, HySparse2 beats HySparse by 11.30 mean MRCR-v2 and 19.81 RULER-v2 points, with lower AgentPPL/LongPPL at all evaluated lengths up to 256k.
- At 1M tokens, HySparse2 cuts prefill FLOPs 2.92× vs HySparse and 5.02× vs Hybrid SWA and holds a 2.69 GB KV cache vs 6.72/12.09 GB (FP8).
Strengths
Clean architectural solution to YOCO-style early exit
Bridging only full-attention layers (§3.2) plus removing the cross-decoder SWA branch in favor of a forced 128-token window (§3.3) eliminates the cascading hidden-state dependency that otherwise blocks prefill exit; §3.5 quantifies the payoff — only 25 of 49 layers must run on the prefill node.
Controlled training comparisons plus a scale-up check
All three 80B-A3B models share the same data mixture and schedules across ~500B-token pretraining and ~100B-token post-training (§4.1); the KV Bridging ablation is repeated at 290B-A8B on 1.8T tokens (Table 5), and bridging is compared against KV Mirror (Fig. 5: RULER 87.65 vs 81.28).
Component-wise ablations with concrete deltas
Large, consistent long-context gains
Honest trade-off accounting
Weaknesses
Head-configuration confound
No statistical treatment
Internal, unverifiable benchmarks
FLOPs-only efficiency story
Perplexity proxy for agent quality
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
HySparse2 (Xiaomi LLM-Core; arXiv 2609.26368) is an architecture paper for long-horizon, multi-turn agentic inference. It claims three things: (1) a YOCO-style self-/cross-decoder split in which only full-attention layers are bridged (KV Bridging, §3.2), so every cross-decoder KV cache derives from self-decoder hidden states and prefill can exit after the self-decoder — 25 of 49 layers on the prefill node (§3.5); (2) two refinements over HySparse — token-level instead of block-level sparse selection, and a forced 128-token recent window replacing the separate SWA branch in sparse layers (§3.3); (3) empirical wins on 80B-A3B MoE models: post-training mean MRCR-v2 +11.30 and RULER-v2 +19.81 points over HySparse, 2.92×/5.02× prefill-FLOPs reductions at 1M tokens versus HySparse/Hybrid SWA, and a 2.69 GB KV cache versus 6.72/12.09 GB at FP8 (Fig. 4).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is HySparse2 and what problem does it solve?
HySparse2 is a hybrid sparse-attention language-model architecture from Xiaomi's LLM-Core team, aimed at long-horizon, multi-turn AI agents that must prefill long observations, store compact KV caches, and still retrieve accurately from very long contexts. It combines a YOCO-style self-/cross-decoder split with two-level KV-cache sharing so prefill only needs to run about half of the model (25 of 49 layers).
How much prefill compute and KV-cache memory does HySparse2 save?
On 80B-A3B models at 1M-token contexts with FP8 KV storage, the paper reports 2.92× lower prefill FLOPs than HySparse and 5.02× lower than Hybrid SWA, with a 2.69 GB KV cache versus 6.72 GB (HySparse) and 12.09 GB (Hybrid SWA). These are analytic FLOPs/byte figures, not measured latencies.
How does HySparse2 differ from HySparse?
It adds outer-level KV Bridging between self- and cross-decoders (only full-attention layers are bridged), switches from block-level to token-level sparse selection, and replaces the separate sliding-window branch in sparse layers with a forced window of the 128 most recent tokens. Together these let prefill exit after the self-decoder, which HySparse cannot do.
Is HySparse2 code or a model checkpoint available?
The paper does not mention any code, checkpoint, or benchmark release. Two headline metrics (AgentPPL and Repo Code PPL) are internal Xiaomi benchmarks, so the reported results cannot currently be reproduced or verified externally.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1, 3, 4, and 5 are axis/legend text only; exact per-length values for the MRCR-v2, RULER-v2, AgentPPL, and LongPPL curves are unavailable except those quoted in prose (e.g., RULER-v2 58.45/32.61/35.74 at 256k).
- AgentPPL and Repo Code PPL are internal benchmarks, and the HySparse, KV Mirror, and MiMo-V2 references are 2026 works that cannot be checked from this text.
- KV-cache and FLOPs figures assume FP8 storage at 1M tokens; per-component byte accounting cannot be re-derived from the extracted text.
- The text appears complete through the references, but minor extraction artifacts around figures are possible.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.