The Spoken Wikipedia Presentation Corpus
Thomas Ranzenberger, Steffen Freisinger, Tobias Bocklet, Korbinian Riedhammer
eess.AS · cs.CL · cs.MM · cs.SD
Quick verdict
SWPC pairs Spoken Wikipedia audio with LLM slides (553 h); audio baselines strong; zero-shot slide context fails; gain unproven - fine-tuning untested.
Summary
The Spoken Wikipedia Presentation Corpus (SWPC) extends the Spoken Wikipedia corpora with LLM-generated slides built by a hybrid pipeline: LLM content planning, deterministic 720p rendering, and OCR re-extraction of slide text. It spans 240,605 utterances from 19,231 articles and 553.5 hours of English, German, and Dutch audio with speaker-disjoint splits. Benchmarking seven audio-only systems (Whisper, Canary, Granite Speech, Cohere transcribe, Phi-4, Qwen2.5-Omni, Voxtral), cohere-transcribe-03-2026 is best on average with 10.23% micro-WER and 6.48% micro-CER; Voxtral leads English and Phi-4 German, while Dutch is hardest. Zero-shot prompting with slide Markdown or images fails through insertion errors from copied slide text, so the authors conclude the task needs fine-tuning. Key limitations: slides are entirely synthetic and unvalidated by humans, the whisper-based QC filter favors the whisper baseline, and contextual-prompting results lack a results table; the abstract's strong cross-modal potential claim remains untested.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- SWPC is the first large, open, multilingual (EN/DE/NL) benchmark for slide-conditioned ASR, pairing 240,605 utterances and 553.5 hours of speech with LLM-generated slides and OCR-derived Markdown.
- On audio-only inputs, cohere-transcribe-03-2026 achieves the best average performance with 10.23% micro-WER and 6.48% micro-CER, with English easiest, then German, then Dutch.
- Zero-shot prompting of omni models (Phi-4-multimodal-instruct, Qwen2.5-Omni-7B) with full slide Markdown or slide images fails due to insertion errors from copied slide text and implicit OCR, with some conditions exceeding 90% WER/CER.
- Effective use of SWPC's slide context requires task-specific adaptation or fine-tuning, which is deferred to future work.
Strengths
Scale and licensing make slide-conditioned ASR reproducible for the first time at this size
SWPC provides 240,605 utterances, 19,231 articles, 46,476 unique slides, 553.5 hours across English/German/Dutch (Table II), with speaker-disjoint 80/10/10 splits — far beyond the small, mostly English talk sets cited in Sec. II, and released openly on HuggingFace with transcripts, slide images, slide Markdown, illustration prompts, and illustrations.
Fully documented, largely deterministic corpus pipeline
Sec. III specifies concrete parameters: ECAPA-TDNN embeddings with a 0.72 cosine threshold for speaker identity, constrained-JSON slide planning with mistralai/Mistral-Medium-3.5-128B, category-to-theme/layout mapping (Table I), text-prohibited FLUX.2-klein-4B illustrations, LightOnOCR-2-1B Markdown extraction, and whisper-based filtering of misaligned utterances.
Honest negative result on zero-shot contextual prompting
Transparent, replicable evaluation protocol
Weaknesses
Central contextual-prompting results are not tabulated
QC filtering creates circular bias toward the whisper-large-v3 baseline
Synthetic slides plus read speech limit ecological validity, and the cross-modal claim is untested
Corpus validation is thin and contains a numerical inconsistency
No contextual-biasing baseline isolates the context format
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
SWPC is presented (Secs. I, III) as the first large, open, multilingual benchmark for slide-conditioned ASR: 240,605 utterances, 19,231 articles, 46,476 unique slides, and 553.5 hours of English, German, and Dutch speech (Table II), built by extending the Spoken Wikipedia Corpora with fully synthetic slides. Three contributions are claimed: the open benchmark, the released artifacts (audio, transcripts, slide images, slide Markdown, illustration prompts, illustrations), and ASR/SLM/omni baselines in audio-only plus zero-shot contextual settings (Markdown text or slide image). The central empirical findings are that cohere-transcribe-03-2026 is best on average audio-only (10.23% micro-WER, 6.48% micro-CER), that error rates grow EN < DE < NL, and that zero-shot contextual prompting fails for both tested omni models. The abstract's further claim — a "strong potential to improve recognition through cross-modal context" — is not supported by any experiment in the paper, since no fine-tuning study is included (deferred in Sec. V).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the Spoken Wikipedia Presentation Corpus (SWPC)?
SWPC is a public dataset for speech recognition research. It takes the Spoken Wikipedia audio corpora (people reading Wikipedia articles aloud) and automatically adds presentation slides for each section, generated by a large language model and rendered to 720p images. It covers English, German, and Dutch: 240,605 utterances from 19,231 articles, about 553.5 hours of audio, split into speaker-disjoint training, development, and test sets. Each row contains audio, transcript, slide image, OCR-extracted slide text, and the generated illustration.
Which model transcribes SWPC best, and what are the error rates?
On audio-only input, the commercial cohere-transcribe-03-2026 service achieved the best average results: 10.23% micro word error rate (WER) and 6.48% micro character error rate (CER) across the three languages. English was easiest, then German; Dutch was hardest for every system. Among open-weight models, Voxtral-Small-24B performed best on English (6.32% WER) and Phi-4-multimodal-instruct on German (9.99% WER).
Do the slides actually improve speech recognition in this paper?
Not in the experiments shown. When the authors gave multimodal models the slide text (Markdown) or the slide image together with the audio via zero-shot prompts, performance collapsed — models copied slide text, performed implicit OCR, or produced non-transcription output, exceeding 90% WER in some conditions. The authors conclude that exploiting slide context will require task-specific fine-tuning, which is left to future work.
Are the slides real presentation slides?
No. All slides are synthetic: an LLM (Mistral-Medium-3.5) plans title, bullets, and takeaway from the article section, a rule-based renderer produces the 720p image, and a diffusion model (FLUX.2-klein) draws a text-free illustration. The corpus therefore measures recognition with idealized slide context and does not capture real presentation acoustics or slide-design variability.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 and 2 are described only via captions in the extracted text; the rendered example slide and pipeline diagram could not be inspected.
- A contextual-prompting results table may exist in the original PDF but was not captured by text extraction — the qualitative-only reporting judgment is based on the extracted prose.
- Evaluation details such as decoding configuration and possibly per-condition numbers may be truncated relative to the published PDF.
- The HuggingFace dataset structure referenced in a footnote could not be verified from the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.