Quantifying Consonant Contributions to Word Intelligibility via Acoustic Masking
Eunjung Yeo, Kwanghee Choi, Krupaben Kothadia, Visar Berisha, Julie M. Liss, David R. Mortensen, David Harwath
cs.CL
Quick verdict
ASR-based silence masking (MMR) ranks consonant contributions across 4 languages; freq/FL correlations are internally consistent but await human validation.
Summary
The paper introduces the mask-induced misrecognition rate (MMR): each consonant in an isolated content word is replaced with silence (with a 25% adjacent-phoneme margin), and a trial counts as misrecognition if an ASR model no longer outputs the reference word. MMR aggregates failure proportions per consonant, computed on typical speech from Common Voice v22.0 test sets (English, Spanish, German, Czech) plus TIMIT as a second English corpus, using three ASR families: MMS (encoder-only), Whisper (encoder-decoder), and Qwen3-ASR (LLM-based). Phoneme frequency and functional load (unweighted minimal-pair counts via Epitran/PanPhon; lexicon restricted to words at 10+ per million tokens) are correlated with MMR via Spearman and partial Spearman correlations. Marginal correlations are weak or inconsistent, but the two factors correlate with each other at rho=0.80-0.95, so partial correlations are decisive: frequency is negatively related to MMR (-0.37 to -0.80) and functional load positively (+0.27 to +0.73), mostly significant; Czech is the exception. Cross-language MMR rankings agree only partially (Spanish correlates significantly with all others at 0.63-0.76; English-German and English-Czech mostly do not reach significance), indicating language-specific contribution. Word-initial masking is more disruptive in English and German but not uniformly. Main limitations: silence is only an upper bound on contribution, no human perceptual validation, single-consonant masking ignores cluster interactions, and correlations rest on roughly 12-17 consonants per language.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 4
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Masking one consonant at a time with silence and scoring ASR misrecognition (MMR) provides a scalable, language-agnostic measure of consonant contribution to word intelligibility.
- After partialling out their mutual correlation, phoneme frequency is negatively associated with MMR while functional load is positively associated (Table II, Blocks C and D).
- Consonant contribution rankings are language-dependent: cross-language MMR agreement is only partial, with Spanish correlating significantly with all other corpora but English-German and English-Czech agreement not reaching significance (Fig. 4).
- The pattern holds across three ASR architectures, including encoder-only MMS, so it is unlikely to stem primarily from language-model priors.
- MMR can serve as a language-level prior for prioritizing intervention targets in motor speech disorders, with speaker-specific factors overriding the ranking.
Strengths
Carefully isolated probe design
Block 1 baseline validation (Fig. 1) retains only correctly recognized words so misrecognitions are attributable to masking; words are decoded in isolation to avoid the sentence-context confound the authors explicitly cite from prior phoneme-masking work [25]; homophones are scored correct; a 25% adjacent-phoneme margin suppresses coarticulatory leakage.
Architecture triangulation against the LM-prior objection
All three ASR families (encoder-only MMS, encoder-decoder Whisper, LLM-based Qwen3-ASR) show the same partial-correlation signs in Table II; the paper argues the negative frequency effect would strengthen from MMS to Qwen if recognizer priors drove it, and it does not — a thoughtful artifact check.
Confronts freq-FL collinearity head-on
Internal cross-corpus consistency check and honest cross-language reporting
Transparent limitation framing
Weaknesses
No human validation of the proxy
Small-n rank statistics without correction
Per-model trial sets confound model comparison
Researcher degree of freedom in headline figures
Post-hoc interpretation and unexamined Czech failure
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This Interspeech-style paper introduces the mask-induced misrecognition rate (MMR): silence one consonant at a time in an isolated content word, re-decode with ASR, and score a trial as a misrecognition if the reference word is no longer output. Aggregating per consonant yields a scalable contribution ranking. Three claims are advanced: (1) MMR is language-agnostic and applicable wherever a forced aligner and multilingual ASR exist; (2) after partialling out their mutual correlation, phoneme frequency correlates negatively with MMR while functional load (FL) correlates positively; (3) rankings are language-dependent, agreeing only partially across English, Spanish, German, and Czech. A clinical motivation — prioritizing intervention targets in motor speech disorders — frames the work but is not directly tested.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the mask-induced misrecognition rate (MMR)?
For each consonant, MMR is the proportion of test words in which silencing that consonant's acoustic interval causes an automatic speech recognition model to stop recognizing the word correctly. It is proposed as a scalable estimate of how much that consonant contributes to a word being understood, replacing costly listener studies.
Why does the paper use ASR instead of human listeners?
Perceptual studies require native listeners and stimuli for every phoneme in every language, which does not scale. Prior literature reports that ASR word error rate correlates with clinician-rated intelligibility, so the authors use multilingual ASR models to measure consonant contributions across four languages cheaply. Human validation remains future work.
What did the study find about phoneme frequency and functional load?
The two factors are strongly correlated with each other (rho=0.80-0.95). After partialling each out, more frequent consonants were less disruptive when masked (partial rho -0.37 to -0.80), while consonants carrying more lexical contrast, i.e., higher functional load, were more disruptive (partial rho +0.27 to +0.73). Most results were statistically significant except for Czech.
Do consonants contribute equally to intelligibility in different languages?
No. MMR rankings across English, Spanish, German, and Czech agreed only partially: Spanish correlated significantly with every other language (rho=0.63-0.76), but English rankings did not significantly match German or Czech, supporting the conclusion that consonant contribution is language-specific rather than universal.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- IPA symbols in figure axis labels are garbled by PDF extraction (e.g., 'S' for the voiceless palatal fricative, 'ts' for the alveolar affricate), so per-consonant identities in Figures 2, 3, and 5 are read approximately; no numeric values were taken from garbled labels.
- Exact p-values and the per-language consonant counts entering each correlation are not tabulated in the text (only significance stars are shown); the ~12-17 phoneme counts are inferred from the labeled points in Figures 2 and 3.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.