Neyshekar: An Open Persian Read-Speech Corpus for Automatic Speech Recognition
Ahmad Amirivojdan, Farzad Nadiri, Abolfazl Alizadeh, Shaghayegh Yaraghi
cs.CL · cs.SD · eess.AS
Quick verdict
Open 99h Persian read-speech corpus; matched-budget ASR tests support in-domain WER gains, but single-rater labels and 82% LLM-generated prompts limit scope.
Summary
Neyshekar is an openly licensed (CC0) Persian read-speech corpus: version 6 comprises 62,279 validated recordings (99.02 hours) from 190 contributors covering 34,541 prompts. The prompt pool mixes human-written text (18%) with LLM-generated material (82%, from Claude Sonnet 4.5, GPT-4.5, GPT-5.1), normalized with the shekar library; each submission passed a six-criterion review, with release labels assigned by a single principal validator (a 4-rater study on 300 clips gives Fleiss' κ=0.679, Gwet's AC1=0.900). Compared with Persian Common Voice under identical processing, clips are longer (5.72s vs 3.93s mean), more lexically diverse at matched tokens, and 24.44% are auto-labeled informal. Fine-tuning Whisper small and XLS-R-300M on duration-matched ~32h subsets (3 seeds) reduces in-domain WER by 9.46 and 11.55 points respectively versus Common Voice training, with gains confirmed on a text-disjoint subset (777 clips) and the independent 344-clip PSRB sample (~8 points). Update-matched, mixture, and scaling controls show benefits of more data but no consistent complementarity of mixing. Main limitations: single-rater release labels, unvalidated automatic register/NER annotations, a small contributor base (test split: 30 speakers), 82% generated prompts whose naturalness is unmeasured, and fine-tuned small models still trailing zero-shot Whisper large-v3 on PSRB.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Training Whisper small and XLS-R-300M on ~32h duration-matched Neyshekar subsets reduces in-domain WER by 9.46 and 11.55 points respectively versus Common Voice training, with the advantage holding on a 777-clip text-disjoint subset and the independent 344-clip PSRB sample (~8 points).
- In-domain gains survive both update-matched budgets and contributor-level (not just utterance-level) bootstrap resampling.
- Neyshekar offers longer clips (5.72s vs 3.93s mean), more word types at 626,370 matched tokens (29,520 vs 24,973), and higher informal/entity coverage than validated Persian Common Voice under identical processing.
- Corpus mixing yields no consistent complementarity: a Common Voice gain for Whisper comes at an in-domain cost, and XLS-R favors Neyshekar-only training across tests.
Strengths
Disciplined, budget-controlled experimental design
Duration matching to within one second on hash-verified frozen manifests (Table 7), replication at a common 1,425-update budget (Table 10), and a 64h mixture-vs-only control at 2,382 updates where the mixture is exactly the union of the two 32h subsets (§5.8) — composition effects are separated from data-volume effects, which most corpus papers skip entirely.
Statistical reporting well above corpus-paper norms
Per-seed paired percentile bootstrap CIs from 20,000 resamples (Tables A2, A5, A6), explicit separation of seed SD from test-sampling uncertainty (§4.4), and contributor-clustered resampling on the 30-speaker test (Table A8) — every in-domain WER interval stays clear of zero under both resampling units.
Auditable partitioning and honest overlap reporting
Honest boundary-setting
Reproducibility infrastructure
Weaknesses
Release labels rest on a single principal validator
Unvalidated automatic annotations underpin the headline features
Thin contributor base
Prompt pool dominated by unmeasured LLM text
Utility bounded to small-model adaptation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Neyshekar v6 is a CC0 Persian read-speech corpus: 62,279 accepted clips, 99.02 hours, 190 contributors, 34,541 prompts (Table 3). Its pitch versus Persian Common Voice: longer clips (5.72s vs 3.93s mean), more word types at 626,370 matched tokens (29,520 vs 24,973; Table 4), 24.44% of clips auto-labeled informal, and NER presence in 23.7% of clips. The evaluation claims (i) in-domain WER gains of 9.46 (Whisper small) and 11.55 points (XLS-R-300M) over duration-matched CV training at ~32h/three epochs (Table 8); (ii) transfer to a 777-clip text-disjoint subset and the 344-clip PSRB sample (~8 points); (iii) no consistent mixture complementarity (Tables 12-13). Claim scope is hedged carefully throughout, which is rare and welcome.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
How large is the Neyshekar Persian speech corpus?
Version 6 contains 62,279 validated recordings totalling 99.02 hours from 190 contributors, covering 34,541 distinct prompts. It is released under the CC0 license on Zenodo (concept DOI 10.5281/zenodo.18073632) with code on GitHub.
How does Neyshekar compare with Persian Common Voice?
Under identical processing, Neyshekar clips are longer (5.72s vs 3.93s mean), lexically richer at matched token count (29,520 vs 24,973 word types at 626,370 tokens), and 24.4% vs 16.1% of clips are labeled informal by an automatic classifier. However, it has far fewer contributors: 190 IDs (45.6 effective) versus Common Voice's 4,338 IDs (70.1 effective).
Does training on Neyshekar improve ASR accuracy?
Yes, for the tested setups: fine-tuning Whisper small and XLS-R-300M on ~32h duration-matched subsets cut in-domain WER by 9.46 and 11.55 points versus Common Voice training, with roughly 8-point gains on the independent PSRB sample. Caveats: fine-tuned small models still trail zero-shot Whisper larger models there, and mixing the two corpora showed no consistent benefit.
Is the Neyshekar data human-verified?
Acceptance used a six-criterion rubric applied by a single principal validator, with a separate four-annotator agreement study on 300 clips (Fleiss κ=0.679, Gwet AC1=0.900). The informal-register and named-entity labels, however, are produced automatically and were not human-validated.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-5 and A1 are referenced but not visually analyzable from the extracted text; judgments rely on tables, captions, and inline numbers.
- The abstract in the extraction is truncated mid-sentence ('reduced by 9.5 p...'); headline numbers were taken from the body tables instead.
- Some page-break artifacts exist in the extraction (e.g., stray digits before section headers), but tables appear complete and internally consistent.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.