Hidden in Plain Sight: The Overlooked Significance of Canonical Elements for Extreme LLM Sparsity
Hyeondo Jang, Kwanhee Lee, Dongyeop Lee, Namhoon Lee
cs.LG
Quick verdict
Revives OBD saliency + gradual pruning; credible budget-matched SOTA at 95-99% LLM sparsity, but reasoning collapse and single-run evidence temper it.
Summary
The paper argues that extreme unstructured sparsity (95–99%) in LLMs is attainable by properly combining classical pruning ingredients: second-order saliency in which the diagonal Fisher is approximated at zero cost from Adam's second-moment states, global weight ranking via distributed binary search, a cubic gradual sparsity schedule, and a warmup–decay learning rate coordinated with the sparsity ramp (framework BEST). Evaluated on LLaMA-2-7B/13B, Qwen3-4B/8B, and Qwen2.5-32B with 524M SlimPajama continued-training tokens, BEST reaches WikiText-2 perplexity 13.48 at 95% and 19.67 at 99% on LLaMA-2-7B, versus 18.75/35.53 for the strongest retrained one-shot baseline (SparseGPT+) and 38.91/55.94 for prior SOTA ELSA; under ELSA's own 67M-token budget, BEST‡ still wins (24.34 vs 38.91 at 95%). The 95%-sparse 7B model shows 3.23× decoding speedup and 6.21× memory savings on an RTX 3090 SpMV kernel. OPT-125M ablations support each component. Key limitations, largely acknowledged by the authors: average zero-shot accuracy drops sharply (59.67 to 39.71 at 95%), GSM8K collapses from 87.64 to 28.05 already at 70% sparsity, all results are single runs without variance reporting, speedup depends on a dedicated sparse kernel, and wins over small dense models are narrow.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Combining second-order saliency (Adam-state Fisher proxy), global thresholding, cubic progressive sparsification, and coordinated LR retraining (BEST) pushes pretrained LLMs to 99% unstructured sparsity while beating prior SOTA.
- On LLaMA-2-7B, BEST achieves WikiText-2 perplexity 13.48 at 95% and 19.67 at 99% sparsity, surpassing SparseGPT+ (18.75/35.53) and ELSA (38.91/55.94).
- BEST outperforms ELSA even under ELSA's own matched 67M-token C4 budget (BEST‡: 24.34 vs 38.91 at 95% sparsity).
- Adam's running second moment is an effective zero-cost diagonal Fisher proxy for pruning saliency, validated on OPT-125M (Table 10).
- The 95%-sparse LLaMA-2-7B model delivers 3.23× decoding speedup and 6.21× memory savings on an RTX 3090 with the MACKO SpMV kernel.
Strengths
Budget-matched SOTA comparisons
Table 5 reports tokens/GPUs/wall-clock for every comparison: BEST‡ under ELSA's exact setting (C4, 67M tokens) beats ELSA in less time (1.74h vs 1.87h, PPL 24.34 vs 38.91 at 95%), and BEST beats SparseGPT+ at equal 524M-token budget (13.48 vs 18.75) — removing the standard 'just train longer' objection
Unusually complete component ablations
Table 4 (saliency × comparison group), Table 10 (Adam-state vs recomputed Fisher: no consistent gain even with 2048 samples), Table 11 (global vs ATP allocation), C.1 warmup sweep, C.2 cubic vs linear schedules, and Figure 8 extending the progressive-vs-one-shot ablation to all sparsities on LLaMA-2-7B
Practical validation beyond perplexity
Candor about failure modes
Breadth of evaluation
Weaknesses
Zero-shot degradation contradicts the headline framing
Reasoning capability collapses early
Single-run results without variance
Unreported hyperparameter tuning protocol
Ablations confined to a small model at moderate-high sparsity
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv 2609.06557) attacks unstructured post-training pruning of LLMs in the 95–99% sparsity regime, where one-shot methods (SparseGPT, Wanda) collapse and prior continued-training SOTA (ELSA, Lee et al. 2026) stalls. The central thesis: the perceived sparsity ceiling is a paradigm artifact, and 'elementary' ingredients — second-order saliency (LeCun et al., 1989), progressive sparsification (Zhu and Gupta, 2017), and retraining coordinated with the sparsity ramp — suffice for a new SOTA when scaled properly. Headline numbers (Table 1, abstract): LLaMA-2-7B WikiText-2 perplexity 13.48 at 95% and 19.67 at 99% (dense 5.47), against 18.75/35.53 for SparseGPT+ and 38.91/55.94 for ELSA (Table 12); plus 3.23× decode speedup and 6.21× memory savings at 95% (Figure 3a). The 'hidden in plain sight' framing is fair: there is no new theory; the value lies in coordinated design and unusually honest budgeting.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is BEST pruning for LLMs?
BEST 是一个后训练剪枝框架，它结合了二阶显著性分数（利用 Adam 优化器状态对对角 Fisher 矩阵进行零成本近似）、全局权重排序（通过分布式二分搜索计算）、立方渐进稀疏调度，以及在继续训练期间与稀疏化坡度相协调的学习率预热-衰减策略，实现了 95–99% 的非结构化稀疏度。
How sparse can LLaMA-2-7B get without losing quality?
根据该论文，在 95% 的稀疏度下，它达到了 WikiText-2 困惑度 13.48（稠密模型：5.47），在 RTX 3090 上实现了 3.23 倍的解码加速；但多项选择零样本平均准确率从 59.67 下降至 39.71，且 GSM8K 数学推理能力在仅 70% 的稀疏度下就从 87.64 骤降至 28.05，因此质量限制是真实存在的。
Does extreme sparsity actually speed up LLM inference?
只能通过专用的稀疏内核实现：在使用 MACKO SpMV 内核（批次大小为 1）的 RTX 3090 上，95% 稀疏度的模型解码速度为每秒 176.18 个 token，而稠密模型为 54.47 个（提速 3.23 倍），且内存占用减少了 6.21 倍；标准硬件缺乏对非规则稀疏性的原生支持，因此结构化方法在通用加速方面仍然更容易实现。
Is the code for this paper available?
是的 — 附录 B.1 指向了 github.com/LOG-postech/hidden-in-plain-sight，并提供了完整的超参数表（表 6-8）；不过，每个 7B 的运行在 4 张 A100 级别的 GPU 上大约需要 10 个 GPU 小时。
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.