The Public Discourse Corpus (PDC): A Speaker-Attributed Dataset for Valence and Epistemic Modality with Target Speaker Participation
cs.CL
Quick verdict
Ambitious 186k-sentence speaker-attributed valence+modality corpus with unusually honest validation disclosure; core speaker-attribution accuracy unverified.
Summary
The paper introduces the Public Discourse Corpus (PDC), presented as the first dataset of public-figure interview speech jointly annotated for affective valence and epistemic modality on speaker-attributed utterances. Built by retrieving YouTube videos with yt-dlp, it covers 998 videos from 100 speakers across seven professional domains, yielding 186,642 sentences (3.1 million words). A five-category Target Speaker Participation (TSP) taxonomy filters out content about a speaker that lacks speech by them, with inter-annotator agreement κ=0.616 on a 188-video sample; an audio-first pipeline (faster-whisper large-v3 + pyannote 3.1) separates target-speaker turns, DeepSeek-V4-Flash labels every sentence zero-shot, and GPT-5.5 re-annotates a stratified 1% sample (N=1,863; κ=0.63 valence, 0.53 modality). An illustrative analysis finds positive statements more often emphatic than negative ones (pooled φ 0.191 vs 0.140). The paper is unusually candid about limitations: the released local pipeline's speaker-attribution accuracy has not been human-validated (the 60-utterance ground truth applies to AssemblyAI, not the local pipeline), all labels come from a single LLM, a 400-sentence human gold standard is pending, and coverage is English-only YouTube. Corpus, tools, pipeline, and validation sample are released. Overall: a genuinely useful, well-documented resource whose core accuracy claims await the promised human validation.
Review complete
Assessment at a glance
5
risk flags identified
● high 2
● medium 2
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- PDC is the first corpus jointly annotating affective valence and epistemic modality on speaker-attributed public-figure interview speech, spanning 998 videos, 100 speakers, seven domains, and 186,642 sentences.
- The five-category Target Speaker Participation taxonomy is a reusable solution to the 'content about vs. speech by' problem, with documented reliability (κ=0.616 five-way, κ=0.690 binary).
- Audio-based diarization is necessary for reliable target-speaker separation: text-only LLM classification systematically fails (guest ratios clustered near 100%), while the audio route achieved 91.7% utterance-level accuracy (κ=0.82) against a 60-utterance human ground truth.
- The local faster-whisper + pyannote pipeline is transcript-consistent with a commercial alternative (Jaccard 0.706, 84.1% word overlap on 212 files), though its attribution accuracy is not yet human-validated.
- Valence and modality are largely orthogonal, with positive statements more often emphatic than negative ones (pooled φ 0.191 vs 0.140), presented as descriptive only.
Strengths
Genuine resource gap addressed
Table 1 correctly shows no existing corpus (CMU-MOSEI, IEMOCAP, MELD, MPQA, FactBank) combines valence and modality on the same speaker-attributed utterances of natural discourse; PDC fills this with 186,642 sentences from 100 speakers across 7 domains.
TSP taxonomy is a reusable, documented method
§3.2 gives a five-category taxonomy with explicit decision rules (§3.2.2), a persuasive design rationale (§3.2.3), and IAA on a 188-video sample: κ=0.616 five-way, κ=0.690 binary, with a full confusion matrix (Table 8, diagonal 132/188 = 70.2%) and disagreement breakdown.
Exemplary candor about evidence scope
Internally consistent, complete release
Motivating abstinence from text-only diarization is well evidenced
Weaknesses
The dataset's defining property (speaker attribution) is not validated on the released pipeline
All 186,642 labels come from one LLM with agreement-only evidence
Sentence-level annotation without conversational context
Severe domain imbalance undermines the illustrative correlation analysis
Documentation arithmetic does not fully reconcile with its own confusion matrix
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper introduces the Public Discourse Corpus (PDC): 998 YouTube interview videos from 100 public figures across seven professional domains, segmented into 186,642 sentences (3,113,156 words), each labeled zero-shot for affective valence (positive/neutral/negative) and epistemic modality (emphatic/hedged/neutral). Three claims are advanced: (1) PDC is the first corpus jointly annotating both dimensions on speaker-attributed utterances (Table 1); (2) Target Speaker Participation (TSP), a five-category taxonomy verifying that retrieved videos contain speech *by* the intended speaker rather than content *about* them, is a reusable methodology with κ=0.616 on a 188-video sample; (3) an audio-first diarization pipeline (faster-whisper large-v3 + pyannote 3.1) is necessary for reliable attribution, text-only LLM classification having failed systematically. An illustrative analysis (§4.3) reports positive statements couple more strongly with emphatic modality than negative ones (pooled φ 0.191 vs 0.140).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the Public Discourse Corpus (PDC)?
PDC is a public dataset of 998 YouTube interview videos from 100 public figures (economists, central bankers, investors, geopolitical analysts, commentators, politicians, tech leaders) across seven professional domains. It contains 186,642 sentences (~3.1 million words), each annotated with an affective valence label (positive/neutral/negative) and an epistemic modality label (emphatic/hedged/neutral), released as three joinable JSONL files on Hugging Face.
How reliable are the PDC annotations?
The video-level speaker-presence filter (TSP taxonomy) shows inter-annotator agreement of κ=0.616 for five categories and κ=0.690 for the include/exclude decision. Sentence labels were produced by one LLM (DeepSeek-V4-Flash); GPT-5.5 agreed on 75.9% of valence labels (κ=0.63) and 72.5% of modality labels (κ=0.53) on a 1,863-sentence stratified sample. Human gold-standard validation of the labels was still in progress at publication, and speaker attribution in the released local pipeline had not yet been human-validated — the authors state this openly.
How is PDC different from CMU-MOSEI, IEMOCAP, or MELD?
Those corpora annotate sentiment on YouTube monologues or emotion in acted/scripted material. PDC instead provides two orthogonal dimensions — valence and epistemic modality — on natural interview speech, with known speaker identity, seven domain labels, and video metadata. This enables questions such as whether central bankers hedge more when giving negative forecasts, which single-dimension corpora cannot support.
Is the PDC code and data publicly available?
Yes. The annotated corpus is on Hugging Face (ictchenbo/public-discourse-corpus) as three JSONL files (annotations, speakers, videos); the full construction pipeline (collection, faster-whisper + pyannote diarization, sentence splitting, LLM annotation) and the web-based annotation tools are open-source on GitHub under a research-use license. Only text transcripts and labels are redistributed, not video or audio.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Only the extracted main text was reviewed; Figure 1 exists solely as text and no figures/appendices were analyzable.
- Supplementary artifacts (annotation guidelines, datasheet, Hugging Face and GitHub releases) were not inspected; release completeness is taken from the paper's claims.
- The paper cites model names (DeepSeek-V4-Flash, GPT-5.5) and 2026 arXiv self-references that could not be independently verified without network access.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.