Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures
Se Un Park, Yutae Kim, Junyoung Park
eess.AS · cs.LG · cs.SD
Quick verdict
Same bit width, different outcomes: cross-architecture TTS PTQ study with real-kernel validation and blind tests; strong empiricism, proxy-metric caveats.
Summary
This empirical study asks why identical post-training quantization settings produce wildly different quality outcomes across text-to-speech systems. Under a single protocol, the authors quantize three core models (Supertonic V3, OmniVoice, Kokoro) with full weight/activation ablations of eight replication models, then apply a staged sensitivity procedure blind to two held-out models (Chatterbox, VoxCPM). Four-bit per-channel weights cost 2.8 UTMOS on Supertonic but only 0.07 on Kokoro; per-tensor scaling can be catastrophic even at 8 bits (Kokoro −3.15, WER 0.030→0.261). Component ablations show the sensitive component is model-specific: Supertonic's vocoder alone loses 2.87 UTMOS, and both preregistered blind predictions from model class were wrong — the procedure instead identified the flow-matching decoder in each held-out model. Per-layer GPTQ recovers sensitive components to within 0.1 UTMOS (F5-TTS −0.07, Kyutai −0.08). Real kernels (ONNX MatMulNBits, torchao int4) reproduce simulated orderings at hardware-dependent cost: on a Mac mini M4 Pro, 4-bit Supertonic runs at 0.60× fp32 latency while int8 is slower. Nine configurations meet a strict quality-preserving criterion. Limitations: UTMOS is an English-trained proxy (a 12-rater informal test validates only severe degradation), bootstrap intervals lack multiple-comparison correction, both blind models share codec-token/latent-patch designs, and text is limited to 200 FLORES-200 sentences.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The same PTQ bit width yields different TTS quality outcomes because the sensitive component is model-specific (e.g., W4 per-channel costs 2.8 UTMOS on Supertonic vs. 0.07 on Kokoro).
- The sensitive component cannot be reliably predicted from model class, as demonstrated by two preregistered blind predictions (Chatterbox, VoxCPM) that both failed.
- A staged ablation procedure locates the sensitive component, and per-layer GPTQ or a higher-precision island restores it to within 0.1 UTMOS of fp (F5-TTS −0.07, Kyutai −0.08).
- Per-tensor scaling can cause severe degradation even at 8-bit weights or activations (Kokoro −3.15 UTMOS at W8; Supertonic vocoder A8 −2.98 at WER 1.30).
- Real int8/int4 kernels reproduce simulated quality orderings, but latency and energy are hardware-dependent, so every configuration must be validated on the target runtime.
Strengths
Unprecedented breadth under one protocol
13 TTS systems (3 core: Supertonic V3, OmniVoice, Kokoro; 8 replication models spanning flow-matching, feedforward, and AR codec-token designs; 2 blind held-out models) evaluated with the same 200 frozen FLORES-200 sentences, fixed seeds, and matched-NFE fp baselines — directly addressing the single-system/single-method gap stated in Sec. 1.
Blind preregistered validation with honest negative outcome
Sec. 4.7 fixes the staged procedure before quantizing Chatterbox and VoxCPM; both preregistered class-based predictions (vocoder; VAE decoder/DiT) failed, and the procedure found the flow-matching decoder in each — the strongest possible evidence for the paper's 'not predictable from model class' thesis.
Real kernels, not just simulation
Removes a real confound explicitly
Formalized, consistently applied acceptance criterion
Weaknesses
Quality claims rest on a contested proxy
No multiple-comparison control and test-set selection
Blind validation is thin and architecturally narrow
Under-specified analyses
Contribution is a benchmark plus checklist, not a method
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This 4-page, ICASSP-style empirical study makes three claims: (1) identical post-training quantization (PTQ) settings produce architecturally dependent outcomes across text-to-speech systems; (2) the sensitive component is model-specific and cannot be predicted from model class — demonstrated by two preregistered blind predictions that both failed; (3) a staged ablation procedure plus per-layer calibrated PTQ (GPTQ) can locate and repair the sensitive component, with real kernels confirming simulated orderings at hardware-dependent cost. The scope is unusually broad for a 4-page paper: three core models (Supertonic V3 99M flow-matching, OmniVoice 0.6B masked-diffusion LM, Kokoro 82M feedforward), eight replication models spanning flow-matching, feedforward, and autoregressive codec-token designs, and two models held out and quantized blind (Chatterbox, VoxCPM) — 13 systems under one protocol.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does the paper 'Same Bit Width, Different Outcomes' show?
It shows that applying the same post-training quantization (for example 4-bit weights) to different text-to-speech models produces very different quality losses — from 0.07 UTMOS on Kokoro to 2.8 on Supertonic — because the sensitive component differs per model. It offers a staged ablation procedure that finds the sensitive component (confirmed in two blind tests) and repairs it with per-layer GPTQ or mixed precision.
Is 4-bit quantization safe for TTS models?
Not automatically. The paper finds 4-bit per-channel weights cost 2.8 UTMOS on Supertonic but only 0.07 on Kokoro. Safety depends on which component is quantized — keeping Supertonic's vocoder at 8 bits restores quality to within 0.05 UTMOS — and on per-tensor scaling being avoided, which can ruin quality even at 8 bits.
Does quantization actually make TTS faster?
Not always. On a Mac mini M4 Pro, a 4-bit weight kernel ran Supertonic at 0.60× the fp32 latency, but int8 dynamic quantization was 2.09× slower; GPU results varied by model and device. The authors conclude each quantized configuration must be validated on its target runtime.
Is the code and data for this TTS quantization study available?
Yes. The paper states that code, evaluation sentences, per-utterance scores, timing records, pinned dependency versions, seeds, calibration settings, and the quantization specification of every run are available at github.com/uxfacdev/tts-ptq-map.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–2 are embedded as raw numeric grids in the extraction; exact row/column alignment of the sensitivity map could not be fully verified, though quoted magnitudes match the running text.
- Tables 1–3 were parsed from flattened text; minor cell misalignment is possible.
- 4-page venue format with no appendix in the extraction; extended results (e.g., NISQA-TTS/DNSMOS details on 251 runs) are not inspectable.
- arXiv v1 (Sep 2026); cited software versions (torchao 0.18, ONNX Runtime) may have changed, affecting exact deployment numbers.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.