All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs
Zhixiong Zhao, Zukang Xu, Guangyu Sun, Lifeng Liu, Dawei Yang
cs.LG · cs.AI
Quick verdict
Claims genuine 1-bit PTQ for LLMs via null-space binary factorization + Shapley allocation; strong results, but headline BPW is scoped to linear weights.
Summary
AF1 is a post-training quantization (PTQ) framework that binarizes LLM linear weights under a strict 1.0 effective-BPW budget for those weights. It combines NABF, which builds on DBF's double-binary ADMM factorization with a Hessian-aware surrogate reparameterization, redirects SVID projection residuals into the approximate null space of the fixed factor (folded into scale vectors via a closed-form least-squares fit), and finishes with scale-only KL-distillation reconstruction; and HiSA, which allocates structural budget via layer-level progressive Shapley estimates on NLL degradation and module-level dual directional/magnitude Shapley scores fused by Soft-OR. On seven models from LLaMA-3, Qwen3, and Gemma-3, AF1 at 2.5 BPW outperforms BiLLM, STBLLM, ARB-LLM, and HBLLM (2.51–3.5 BPW) on every model; at 1.0 BPW it reaches 61.80 average zero-shot accuracy and 8.91 WikiText2 PPL on LLaMA-3-70B. Instruction-tuned evaluations (MMLU-Pro, GSM8K, HumanEval) and efficiency tests — roughly 90% memory reduction and 2.5× decoding throughput versus BF16, and 5.9 GPU-hours on LLaMA-2-7B versus 237–1344 hours for QAT — support practicality. Limitations: the 1-bit guarantee excludes embeddings and non-linear layers (whole-model BPW is roughly 1.55), absolute 1-bit accuracy stays about 27 points below BF16 on the 8B model, there is no direct DBF baseline, and no variance or significance reporting.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Existing binarization PTQ methods carry hidden metadata that pushes effective storage to 2.5–3.5 BPW (Appendix A.3), far above the nominal 1-bit target.
- NABF preserves the 1-bit parameterization while suppressing SVID projection error by steering residuals into the approximate null space of the fixed factor and folding corrections into scale vectors (§4.1, A.2).
- HiSA's hierarchical Shapley allocation beats uniform and local-sensitivity allocation at 1.0 BPW (PPL 22.18 vs 35.84 on LLaMA-3-8B, Table 6).
- AF1 at 1.0 effective BPW for target linear weights outperforms 2.5–3.5 BPW binary PTQ baselines (LLaMA-3-70B: 61.80 avg zero-shot, 8.91 PPL) and approaches QAT quality at ~40–228× lower quantization cost (Fig. 4).
- AF1 delivers ≈90% memory reduction and 2.5× decoding throughput versus BF16 across three models (Fig. 5).
Strengths
Rigorous effective-BPW accounting
Appendix A.3 derives full deployable storage for BiLLM/STBLLM/ARB-LLM/HBLLM including bitmaps, salient-column indices and BF16 coefficients, and Eq. 116 gives AF1's exact storage (two binary factors + three BF16 scale vectors) — the comparison axis is explicit and auditable.
Broad, consistent evaluation sweep
Seven models across LLaMA-3/Qwen3/Gemma-3 (Table 1) plus instruction-tuned MMLU-Pro/GSM8K/HumanEval (Table 2); AF1-2.5 is best on all seven models, and AF1-1.0 on LLaMA-3-70B (61.80 avg zero-shot, 8.91 PPL) beats every 2.5–3.5 BPW baseline.
PTQ cost with near-QAT quality
Above-average ablation grid
Honest scoping and disclosure
Weaknesses
'1-bit' excludes non-linear layers
No direct DBF baseline
Outlier-driven aggregate claims
Author-derived baseline BPWs with unstated configs
No variance or significance reporting
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
AF1 targets the gap between nominal and effective bits per weight in binarized PTQ for LLMs: prior methods (BiLLM, ARB-LLM, STBLLM, HBLLM) reach 2.5–3.5 effective BPW once scales, bitmaps, salient-column indices, and sparsity metadata are counted (Appendix A.3; Fig. 1). The paper proposes two mechanisms. **NABF** (§4.1) is a double-binary factorization built on DBF's ADMM-SVID machinery, augmented with a Hessian-aware surrogate space, null-space redirection of SVID projection residuals folded into scale vectors, and a final scale-only KL-distillation reconstruction. **HiSA** (§4.2) allocates structural budget via Monte Carlo Shapley estimates: layer-level on NLL degradation, module-level on dual directional/magnitude distortions. The central claim — genuine 1.0 effective BPW for target linear weights — is consistently scoped in the body text, though the 'genuine 1-bit' headline will read stronger than its scope to casual readers.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'genuine 1-bit' mean in the AF1 paper?
It means the target linear weight matrices are stored at 1.0 effective bits per weight: two binary sign matrices plus three BF16 scale vectors, with no extra masks or group metadata. Embeddings, the LM head, and non-linear layers are not 1-bit; from the reported model sizes (1.56 GB for LLaMA-3.1-8B vs 16.07 GB BF16), the whole model averages roughly 1.55 BPW.
How does AF1 differ from OneBit and LittleBit?
OneBit and LittleBit are quantization-aware training (QAT) methods that need long training runs (237–1344 GPU-hours on LLaMA-2-7B per Fig. 4). AF1 is post-training: it quantizes in about 5.9 GPU-hours and reaches WikiText2 PPL 11.20 at ~1-bit, somewhat behind the QAT references (~9.1–9.7 per the figure) but far cheaper.
Does AF1 match full-precision accuracy at 1 bit?
No. At 1.0 BPW, LLaMA-3-8B averages 45.48% across seven zero-shot tasks versus 72.51% for BF16, and MMLU-Pro drops from 46.64 to 20.12 on LLaMA-3.1-8B-Instruct. AF1 is the strongest reported method at that budget, but a large accuracy gap remains.
Is AF1's code available?
The paper states the code is available at github.com/Kishon-zzx/AF1. It does not state whether pretrained 1-bit model checkpoints are also released.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Numeric labels embedded in figures were garbled by PDF extraction (Figs. 1, 3, 4, 5, 8); the mapping of Fig. 4's remaining PPL values (9.08/9.73) to LittleBit vs OneBit is inferred from context, and Fig. 5's per-config throughput values could not be read reliably.
- Table 3's checkmark pattern was partially garbled in extraction, so the row-to-configuration mapping for its last two rows (HiSA Layer/Module stages) is ambiguous.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.