Quantization Amplifies Determinism, Not Bias: Scale-Dependent Behavioral Effects of Serving-Time Weight Compression
cs.AI
Quick verdict
Pre-specified, well-controlled study: int4 narrows Qwen3-8B answer diversity but not at 14B/32B; credible despite single-family scope and temp-0.4 fragility.
Summary
This preprint asks whether serving-time weight quantization changes not how capable a model is but which valid answers it chooses to give. The author serves three Qwen3 checkpoints (8B/14B/32B) at three precisions (W4A16 AWQ, W8A16 executed weight-only via Marlin kernels, and bf16) on identical hardware, software, and seed sets, collecting roughly 71,000 prompt- and seed-paired completions across two custom, leak-checked prompt batteries (218 country-elicitation prompts plus 80 style controls; 96 brand-blind car-buying scenarios), with analyses pre-specified in three version-controlled waves. At 8B, int4 raises brand-collision probability by 5.1 percentage points (Holm p=.023; reproduced at +4.4pp) and lowers type-token ratio (-0.0113, robust to the length-controlled MATTR-100); the effect is absent on the cultural battery and heterogeneous across prompts. At 14B and 32B no concentration measure is significant, but em-dash rate drifts upward (+0.46 and +0.61 per 1k words, Holm p<=.0024). Pre-specified stereotype-direction tests are null at every scale. Mechanistically, token-level entropy rises (+0.091 bits) while the first-token-measured semantic distribution concentrates (+2.6pp collision). Limitations: one model family, one quantizer per precision, a single domain (US automotive, English), 32B confounded by TP=8 and different hardware, the headline collision effect losing significance at temperature 0.4, and the repository of frozen statistics promised only for camera-ready.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- At Qwen3-8B, int4 AWQ serving increases brand-collision probability by 5.1pp (Holm p=.023, reproduced at +4.4pp) and reduces lexical diversity (TTR -0.0113), narrowing the valid-answer distribution.
- Content-concentration effects are absent at 14B and 32B, where int4 instead produces stylistic drift (em-dash +0.46/1k at 14B, +0.61/1k at 32B, Holm p<=.0024).
- Pre-specified stereotype-direction tests are null at every scale: quantized outputs concentrate on each prompt's modal answer, not on stereotypical answers.
- Under int4 at 8B, token-level decision entropy rises (+0.091 bits) while the directly measured first-token semantic distribution concentrates (+2.6pp collision), so this failure mode cannot be audited from token-level logits alone.
Strengths
Controlled serving-time precision ladder removes the usual confounds
Same official Qwen3 checkpoints at bf16, W8A16 (FP8 checkpoint run weight-only via Marlin) and W4A16 AWQ, one vLLM 0.11.0 stack, identical flags, and shared seed sets with n=20 per prompt per tier (Section 3); prompt-and-seed pairing removes sampling-RNG variance from every contrast, and the regeneration check actively exercises residual batching nondeterminism.
Unusually strict confirmatory discipline
Three-wave declare-then-run protocol in version control (Contributions, item 3); Holm correction within declared families; CI-only quantities explicitly demoted to supporting evidence (Statistical protocol, Tables 1-3); no tests added after inspecting the corresponding data.
Built-in replication and robustness checks
Directly measured mechanism dissociation is a genuine methodological contribution
Transparency about provenance and effect structure
Weaknesses
Single model family and single quantizer per rung
The headline 8B effect is fragile across sampling regimes
The 14B column lacks clean confirmatory status
The not-bias headline partly rests on underpowered tests
Key artifacts not yet available
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This preprint (arXiv:2609.07901, D. Kurtskhalia, Impel) reframes the cost question for weight quantization: instead of capability loss on single-answer benchmarks, it asks whether quantization changes which valid answers a served model produces - a distributional property invisible to argmax evaluation. The motivating distinction (Section 1) between bias (systematic favoritism toward particular brands) and concentration (narrowing the recommended set, hurting tail inventory) is well drawn and commercially concrete.
Four headline claims are staked (Abstract, Sections 1 and 4): (1) at Qwen3-8B, int4 AWQ serving raises brand-collision probability by +5.1pp (Holm p=.023, Table 1; reproduced at +4.4pp on a full regeneration) and depresses lexical diversity (TTR -0.0113, standardized -0.51, MATTR-100 robust); (2) concentration vanishes at 14B/32B, replaced by stylistic drift (em-dash +0.46/1k and +0.61/1k, Holm p<=.0024, Table 2); (3) pre-specified stereotype-direction tests are null at every scale (Table 4); (4) token-level decision entropy rises (+0.091 bits) while the directly measured first-token brand distribution concentrates (+2.6pp collision, Section 4.4) - a novel dissociation implying logit-based audits cannot detect this failure mode.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does 4-bit quantization make LLM outputs more biased?
In this paper, no. Pre-specified stereotype-direction tests were null at 8B, 14B, and 32B: quantized outputs concentrated on each prompt's modal answer rather than on stereotypical answers. What changed at the smallest model was concentration - fewer distinct brands were recommended across repeated samples - which the authors argue requires a different audit than bias.
Is 4-bit quantization really free for 8B models?
Not entirely, according to this study. While capability benchmarks may show little loss, serving Qwen3-8B at int4 (AWQ) measurably narrowed recommendation diversity (brand-collision probability up 5.1 percentage points, Holm p=.023) and lexical diversity (type-token ratio down 0.0113) - damage that argmax-style benchmarks cannot detect.
Does quantization change how larger models like 32B behave?
In this paper, content concentration disappeared at 14B and 32B; instead the authors found stylistic drift - em-dash rates rose by 0.46 and 0.61 per 1,000 words respectively (both Holm p<=.0024). What the models said was unchanged; only how they phrased it shifted.
How can I audit a quantized model for answer concentration?
The paper recommends sampling k answers per prompt and measuring collision probability (the share of samples giving the same first-mentioned entity) or the number of distinct answers. Token-level logit metrics did not detect the semantic concentration, because token entropy actually rose while answer diversity fell.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The abstract as supplied in the task metadata is cut mid-sentence, but the complete abstract appears at the head of the embedded full text, so no abstract content was lost.
- The full 30-row wave-1 table and the frozen statistics CSVs reside in the paper's external repository by the paper's own statement (Appendix A, Reproducibility), not in the provided text; that repository was not examinable for this review.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.