Assessing the Reusability of Public Speech Resources for Low-Resource Languages: A Central Kurdish Case Study
cs.CL
Quick verdict
Careful audit of a Kurdish TTS release vs its paper: stale config, unmarked test split, card overclaims; findings are concrete, fair, and mostly verifiable.
Summary
This single-author review audits how closely the public release of a Central Kurdish text-to-speech system — three F5-TTS-based voices fine-tuned on roughly 35 hours of audiobook and studio speech, published with an LREC 2026 paper by Mohammadamini, Shamsi and Tahon — matches the research it describes. Audio quality is not assessed; the comparison covers model provenance, configuration, evaluation design, licensing, and linguistic documentation. Documented mismatches include: a released configuration file carrying an eight-GPU comment and eleven epochs where the paper reports one GPU and roughly 275,000 training steps; 1,500 held-out test utterances left unmarked inside the released training corpus; an inference-script bug that replaces digit runs of nine or more with a placeholder that is never restored and contains a character absent from vocab.txt; and a model card claiming the 'highest overall subjective score' and 'general-purpose' fitness where the paper says 'competitive' and 'marginally better' (MOS 4.08 vs 4.06, per-category CIs ±0.14–0.19). It further shows the intelligibility metric is instrument-biased — the recognizer that produced the training transcripts scores 0.09 character error rate on real studio speech versus 0.01–0.02 on audiobook speech — and analyzes license provenance, missing speaker-variety metadata, and an unnamed rights holder for the male audiobook. Limitations: the audit is non-experimental; several claims rest on code reading rather than execution, and the auditor's independence from the original team is not established.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 2
● low 1
5 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The LREC 2026 Central Kurdish TTS paper documents its work carefully, but the artifacts most users encounter — model card, configuration file, dataset, evaluation code — diverge from it in ways that mislead reuse.
- The model card's 'highest overall subjective score' and 'general-purpose' claims overstate a marginally better result (MOS 4.08 vs 4.06) that lies well inside the reported per-category confidence intervals of ±0.14–0.19.
- Character error rate evaluation is instrument-biased because the scoring recognizer also produced the training transcripts, and its error on real studio speech (0.09) far exceeds that on audiobook speech (0.01–0.02).
- CC BY-NC-ND terms, inherited from audiobook rights holders and non-commercial F5-TTS weights, allow private but effectively forbid public verification through modified models, and neither provenance is documented.
- The pipeline cumulatively removes regional and register variation (read speech only, G2P defaults, single-pronunciation selection), so the voices represent three narrators' reading styles rather than Central Kurdish as spoken.
Strengths
Artifact-level verification with pinned revisions
All evidence is tied to commits and dates (model repo commit cdca52a, demo commit b5c6a0a, eval repo commit 116761d, upstream F5-TTS commit 9c614e9, all retrieved 8 Sep 2026), and the author actually installed and ran the released TTS4ALL_Eval package, reproducing its shipped reference values across all seven metrics (Section 7).
Fair attribution of restrictions and credit
Section 8 traces CC BY-NC-ND to audiobook rights holders (p. 671 quote) and the non-commercial term to F5-TTS weights, explicitly rejecting misdirected criticism; Section 1 disclaims any misconduct claim and Section 13 systematically catalogs what the release does well (CIs throughout, running eval code, the 35-hour corpus).
Quantified use of the source paper's own tables
Constructive, cost-ranked remedies
Original linguistic analysis absent from the audited paper
Weaknesses
Central behavioral claims never executed
No formal audit methodology
Point-in-time evidence without archives
Auditor independence unstated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a single-author audit (Asadpour, Saarland / Goethe University Frankfurt) of a *different* paper's public release: Mohammadamini, Shamsi and Tahon, LREC 2026, pp. 664–673, which shipped three Central Kurdish TTS voices fine-tuned from pretrained F5-TTS weights on ~35h20m of transcribed speech (three single speakers: 6,044 / 5,572 / 6,055 utterances per Table 1), plus evaluation code. The review explicitly excludes audio-quality assessment. Its central claim is narrower and more interesting than a quality verdict: the *paper* documents the work carefully while the *artifacts* most users touch — model card, config file, dataset, eval code — diverge from it, and in a low-resource language that gap is expensive because there are no alternative resources to check against. No misconduct is alleged; the framing is documentation failure, not fraud, and the review names its evidence standard on sources (Section 1).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What problems did this review find in the Central Kurdish TTS release?
It found a released training config describing eight GPUs and eleven epochs while the paper reports one GPU and roughly 275,000 training steps; 1,500 test utterances left unmarked inside the public training corpus; an inference bug replacing 9+ digit numbers with a placeholder that is never restored (its '<' character is missing from vocab.txt); and a Hugging Face model card claiming the 'highest overall subjective score' and 'general-purpose' fitness where the paper only claims results are 'competitive' (4.08 vs 4.06 MOS, within ±0.14–0.19 confidence intervals).
Can I train on the released central-kurdish-tts4all dataset?
Yes, the ~17,671 utterances (35 hours) are downloadable under CC BY-NC-ND 4.0, but the review warns that the paper's 1,500-sample test set is not marked in the release, so fine-tuning without separating it first would silently train on the paper's evaluation data. The license also forbids redistributing modified versions, which covers models trained on the data under a plain reading.
Why is the CC BY-NC-ND license a problem for this release?
The no-derivatives term means corrected or dialect-adapted models cannot legally be shared: verification must stay private, and a community fix for the documented number bug cannot be published. The review shows the terms were set by audiobook rights holders (and non-commercial terms by the F5-TTS pretrained weights), not chosen by the researchers, and asks them to seek a model-only redistribution exception.
Is the female audiobook voice really the best Central Kurdish TTS system?
Only marginally: 4.08 MOS versus 4.06 for the studio male system, well within the ±0.14–0.19 confidence intervals, and each condition is a single speaker, so 'audiobook' cannot be separated from that narrator's voice and skill. The review argues the model card's 'highest overall score, general-purpose' framing overstates what the paper's own listening test supports.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures (e.g., Figure 1 learning curves) are referenced by description only; no rendered figure exists in the extracted text, so the ~275,000-step claim rests on the paper's own prose.
- The metadata abstract is cut mid-sentence ('will affect'), but the embedded full text ends naturally after its references and 'Not established' list, so no truncation of the paper body is assumed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.