EFQ-Softmax: Exp-Free Quantization for Softmax
Haohui Han, Yuming Wan, Hongni Wang, Pengcheng Xie, Xiaodong Yan, Runqi You, Wencong Zhang
cs.LG
Quick verdict
Exp-free direct E2M1 probability generation for FP4 attention preserves quality vs MXFP4, but language gains are unproven and speed is kernel-stage only.
Summary
EFQ-Softmax targets the probability path of FP4 attention: instead of computing elementwise exponentials of shifted attention scores and then quantizing them, it maps shifted scores directly to block-scaled E2M1 probability codes. Per microscaling block, an exponent-only scale is derived from the local maximum, scores are normalized into a residual log-domain, and a single affine threshold rule with two offline-calibrated parameters (τ, h) emits 4-bit codes; the same operand feeds both the PV numerator and the P·1 denominator of the otherwise-unchanged FlashAttention recurrence. On Qwen3-8B the seven-task zero-shot mean rises from 0.6749 (MXFP4) to 0.6773, still below FP16's 0.6803; Qwen3-VL-8B-Instruct's nine-task mean improves from 0.7826 to 0.8000 with gains on all six grounding splits; WAN2.2 VBench metrics stay comparable to FP16 and MXFP4. On the Ascend 950PR vector unit, the fused kernel cuts vector-stage latency by 40.33% on average (fixpipe only ~0.8%). Key limitations: no seeds, variance, or significance tests; (τ, h) operating points were selected by sweeping on the evaluation benchmarks themselves; the large vision-language gain over a less-quantized baseline is weakly explained; no end-to-end latency, no head-to-head against SageAttention3's probability path, and no code release.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 4
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Directly generating block-scaled E2M1 probability codes from shifted scores can replace the conventional exp-then-quantize path while preserving end-to-end model quality (Sec. I, V).
- A single affine threshold rule with two globally calibrated parameters (τ,h), plus a per-block exponent-only scale, suffices for accurate 8-code probability generation shared by both softmax branches (Sec. II, III-C).
- EFQ-Softmax matches or slightly exceeds MXFP4 on Qwen3-8B (0.6749→0.6773) and Qwen3-VL (0.7826→0.8000) while remaining comparable to FP16 and MXFP4 on WAN2.2 VBench (Sec. IV B–D).
- The fused probability-generation kernel reduces Ascend 950PR vector-stage latency by 40.33% on average across 16K–128K sequence lengths (Table VIII).
Strengths
Method fully specified in closed form
Sec. II gives the scale rule (Eq. 8 with the fixed log(2/9) constant), residual log-domain normalization (Eq. 10), affine code rule (Eq. 11), and Algorithms 1–3 including offline calibration — the math is complete enough to reimplement without the paper's code.
Numerator/denominator consistency handled explicitly
Sec. III-A (Eqs. 12–14) uses one shared block-scaled E2M1 operand in both the P~V and P·1 branches, avoiding the two-approximation bias that silently corrupts online softmax in naive exp-approximation schemes — a correctness point most exp-free softmax papers skip.
Broad end-to-end evaluation with a replication gesture
Honest baseline ordering on language tasks
Weaknesses
Operating points selected on the test benchmarks
No statistical rigor
Anomalous Qwen3-VL win over its own reference
Speedup evidence is narrow and ambiguous
Reproducibility and reporting gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
EFQ-Softmax (arXiv 2609.09721v2, cs.LG) replaces the 'exp-then-quantize' probability path of low-bit attention: instead of evaluating elementwise exponentials of row-max-shifted scores in FP16/FP32 and quantizing the tile to E2M1, it directly generates block-scaled E2M1 probability codes. The mechanism (Sec. II): per microscaling block an exponent-only scale s_b = 2^k_b is derived from the local maximum (Eq. 8, with the fixed constant log(2/9) aligning 6·s_b to e^{M_b}); scores are normalized into a residual log-domain (Eq. 10); a single affine rule c = clip[0,7](floor((z−τ)h)+1) (Eq. 11) with two offline-calibrated parameters (τ,h) emits the codes (Sec. III-C, Alg. 3). Central claims: (1) one shared operand serves numerator P~V and denominator P·1 (Eqs. 12–14) while the FlashAttention recurrence stays intact; (2) end-to-end quality matches or beats MXFP4 on Qwen3-8B and Qwen3-VL and matches FP16/MXFP4 on WAN2.2 VBench; (3) the fused kernel cuts Ascend 950PR vector-stage latency 40.33% on average over 16K–128K sequences.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is EFQ-Softmax?
A low-bit attention method that skips computing exponentials of attention scores: it maps row-max-shifted scores directly to 4-bit E2M1 probability codes via a per-block power-of-two scale and a single affine threshold rule with two calibrated parameters (τ, h), then uses those codes in both the softmax numerator and denominator of FlashAttention-style online attention.
How much faster is EFQ-Softmax?
The paper reports the fused probability-generation kernel's vector-stage latency drops 40.33% on average across 16K–128K sequence lengths on the Ascend 950PR vector unit, with only about 0.8% gain in the fixpipe stage. No end-to-end attention latency or full-model speedup numbers are provided.
Does EFQ-Softmax lose accuracy compared to FP16?
Slightly. On Qwen3-8B the best reported EFQ variant reaches a 0.6773 seven-task zero-shot mean versus 0.6803 for FP16 and 0.6749 for MXFP4. On WAN2.2 text-to-video, VBench metrics are comparable to both FP16 and MXFP4 baselines.
Is code for EFQ-Softmax available?
The paper does not mention any code or artifact release. Reproducing it requires reimplementing the quantization pipeline from the paper's equations, and the kernel-level results require Ascend 950PR hardware.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are caption-only in the extraction; the visual content of Figs. 1–7 (pipeline diagrams, mapping plots, qualitative video frames) could not be assessed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.