Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
Sergii Kozyrev, Davyd Maiboroda
cs.AI
Quick verdict
Shows GDN survives full W4A4 NVFP4 in a 27B hybrid with a solid mechanism study; convincing, though single-model with thin multi-seed statistics.
Summary
The paper quantizes all 496 linear layers of Qwen3.8-27B — including the Gated DeltaNet (GDN) recurrent block and its decay/write-strength gate projections — to NVFP4 W4A4 ("Minima"), challenging the community practice of keeping the recurrent block at 8/16 bits. Under a fixed serving regime (FP8 KV cache, vLLM 0.27.1, one RTX PRO 6000), Minima matches BF16 and two community NVFP4 recipes within seed noise on MMLU-Pro, GSM8K, AIME'25 (4 seeds), GPQA-Diamond, LiveCodeBench, and RULER retrieval to 64K (5-task average −0.52 vs BF16), while being the smallest (17.53 GiB vs 50.13) and fastest at prefill (TTFT@32K 4.03 s vs 6.90 s). A four-part mechanism study explains why: 16-element block scaling localizes residual-stream outliers; softplus/exponential and sigmoid gate parameterizations compress ~11% GEMM error to ~2% output error; the delta rule holds state error at a flat ~12.6% plateau over 32K tokens and erases a state impulse within hundreds of steps; and the end-to-end perplexity gap shrinks with position (+0.081→+0.011 nats). The paper also documents serving-stack pitfalls — a fused-GEMM global-scale mismatch that faked better-than-BF16 perplexity (6.86 vs 10.84), multimodal-path effects, invalid raw-completion harnesses — and shows calibrated FP8 KV scales recover 83% of the long-context KV penalty. Limitations: one model family and format, evaluation only to 32K perplexity/64K retrieval, largely single-run task scores, and the concurrent QAT checkpoint (QUASAR) left unbenchmarked.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Full NVFP4 W4A4 post-training quantization of all 496 linear layers of Qwen3.8-27B — GDN gates included — matches BF16 and two community recipes within seed noise across six accuracy suites while being the smallest (17.53 GiB) and fastest-prefill (4.03 s TTFT at 32K) recipe.
- The gate projections the community protects in BF16 are actually the least quantization-sensitive, because the log-space softplus/exponential and sigmoid parameterizations compress ~11% GEMM error to ~2% output error.
- The delta-rule recurrence bounds injected state noise at a flat plateau (relS ≈ 12.6% over 32K tokens) and erases a state impulse within hundreds of steps — far faster than decay-gate horizons — so quantization error does not compound with context.
- A mismatch between per-module NVFP4 calibration and fused serving GEMMs silently corrupts the GDN gates and can fake better-than-BF16 long-context perplexity (6.86 vs 10.84); a checkpoint-side scale harmonization repairs it.
- Calibrated per-tensor FP8 KV-cache scales are throughput-free and recover 83% of the +0.41 PPL@32K penalty that FP8 KV imposes on the W4A4 model.
Strengths
First true W4A4 of the recurrent block, with released checkpoint
Minima quantizes all 496 linear layers including the GDN gate projections a/b (§3), whereas Unsloth and RadixArk keep GDN/attention at FP8 W8A8 with a/b in BF16; the quantized checkpoint is released on Hugging Face (minima-ai/mnma_qwen3.8_27b_nvfp4), and the concurrent QAT route (QUASAR) is shown to be unnecessary for BF16-level task accuracy.
Causal mechanism chain with quantified evidence at each link
§5 rules out easy inputs (Table 2: GDN inputs as outlier-laden as attention, max/RMS 63.5, kurtosis ~1,564), shows gate projections move output only 2.1%/2.6% despite 11.0%/8.5% GEMM error (Table 3), demonstrates flat state error (relS 12.96%→12.31% from token 256 to 32,768, Table 6) with impulse erasure in 80–1,382 steps vs decay horizons of 44K–62K tokens, and closes with positional NLL decomposition (+0.081→+0.011 nats, −0.053 in final bins, Table 4).
The synthetic-noise arm identifies the mechanism rather than merely correlating
Measurement integrity unusually candid and consequential
Practical payoff is concrete, not just accuracy parity
Weaknesses
Statistical power is thin for the headline equivalence claim
Single model, single size, single format
RULER is saturated and cannot discriminate
Mechanism study samples depth coarsely
The released checkpoint's task scores are inherited, not measured
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether the recurrent half of a hybrid LLM — Gated DeltaNet (GDN) — truly needs precision protection under 4-bit weights-and-activations quantization. Using Qwen3.8-27B (48 GDN layers, 16 attention layers), the authors build "Minima": NVFP4 W4A4 (E2M1 values, E4M3 scale per 16-element block) applied to all 496 backbone linear layers, including the gate projections a (decay) and b (write strength) that every public recipe keeps in BF16. Three claim families: (1) empirical equivalence — Minima matches BF16 and two community NVFP4 recipes (Unsloth, RadixArk) within seed noise on six suites (Table 1: 5-task average 85.10 vs 85.62 for BF16, −0.52) while being smallest (17.53 vs 50.13 GiB) and fastest at prefill (TTFT@32K 4.03 s vs 6.90 s); (2) a mechanistic account (§5) of why; (3) serving-stack and KV-cache findings (§6–7) required to measure any of it correctly. The headline inversion — the "protected" projections are the safest — is well supported.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Minima?
Minima is the authors' NVFP4 4-bit weights-and-activations (W4A4) quantization of the Qwen3.8-27B hybrid LLM. Unlike earlier community 4-bit builds that kept the recurrent Gated DeltaNet block at 8/16 bits, Minima quantizes all 496 linear layers, including the recurrent block's decay and write-strength gates. The quantized checkpoint is released on Hugging Face (minima-ai/mnma_qwen3.8_27b_nvfp4).
Does 4-bit quantization break recurrent or linear-attention layers in hybrid LLMs?
According to this paper, not for Gated DeltaNet in Qwen3.8-27B: a fully W4A4 model matched the BF16 model within seed noise on six benchmarks, and a mechanism study shows that block scaling, gate nonlinearities, and the delta rule's overwrite behavior keep quantization noise at a flat plateau instead of accumulating over long contexts.
Why did earlier quantized versions of this model protect the GDN layers?
Because of the intuition that per-step errors in a recurrence compound over long contexts, especially in the decay and write-strength gates. The paper shows this intuition is backwards for this architecture: the protected gate projections are the least quantization-sensitive, and injected state error stays flat and is actively erased by the delta rule.
How much GPU memory does the W4A4 model need?
About 17.5 GiB for weights, versus 50.1 GiB for the BF16 model, running natively on NVFP4-capable NVIDIA hardware — the paper uses a single 96 GB RTX PRO 6000 with vLLM — which also leaves room for a much larger KV cache on one card.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.