Queer inclusion in speech datasets: An audit and taxonomy of practical tensions
Brooklyn Sheppard, Anaelia Ovalle, Adina Williams, Levent Sagun
cs.AI
Quick verdict
Audits 8 speech datasets, finding 0–1.4% queer speakers, and proposes a 4-way tension taxonomy; descriptive but a solid, timely audit with thin statistics.
Summary
This audit study asks whether mainstream speech technology datasets include queer (LGBTQIA+) voices. The authors audit six English speech datasets — Casual Conversations V2, Common Voice 22.0, Edinburgh Accents, English Dialects, Fairspeech, and L2-ARCTIC — plus two queer-specific corpora (MAGES, PTMV), along six axes: gender annotation options, label distributions, institutional affiliation, ethics review disclosure, recruitment methods, and data access. The headline result is that measurable queer representation spans 0 to 1.4% of speakers (Common Voice: 0 of 94,911 contributors; CCV2: 1.44%; EdAcc: 0.82%), which the authors argue is insufficient for robust disparity measurement. They also surface that Fairspeech excluded non-binary speakers from its release despite collecting some ([35], p. 2). From collection practices they derive a taxonomy of four tensions: scaling versus inclusion, efficiency versus engagement, access versus autonomy, and static categories versus fluid identities. The paper is descriptive: no statistical testing, tiny counts behind percentages (EdAcc's 0.82% is a single speaker), a six-dataset English-only sample, and 'queerness' operationalized solely through gender identity labels — limits the authors largely acknowledge. Its value lies in documentation and conceptual framing rather than generalizable measurement.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Measurable queer representation across six English speech technology datasets is 0–1.4% of speakers, which the authors argue is insufficient for robust disparity measurement.
- Common Voice 22.0 contains zero contributors who openly identify as transgender or non-binary despite its 94,911 speakers, showing that open crowdsourced scaling does not automatically include stigmatized minorities.
- Fairspeech collected non-binary speakers but excluded them from the released dataset to avoid skewed results ([35], p. 2).
- Common AI data-collection customs conflict with participatory values along four tensions: scaling versus inclusion, efficiency versus engagement, access versus autonomy, and static categories versus fluid identities.
- Queer-specific corpora (MAGES, PTMV) demonstrate alternatives: ethics review, community-based recruitment, and free-text gender self-description.
Strengths
Concrete, checkable headline numbers
Table 1 quantifies queer representation per dataset (CCV2 1.44% of ~5,567 speakers; EdAcc 0.82% of 122; Common Voice 22.0: 0 of 94,911; others 0), and the size-ratio claim in §4.1 (CV is 778x and 17x larger than EdAcc and CCV2) is arithmetically consistent with the table.
Documents an under-reported practice
§3 quotes the original Fairspeech paper ([35], p. 2) admitting non-binary speakers were collected but excluded from the released dataset 'to not show skewed results' — a concrete, citable instance of representational erasure, not just an inference.
Audit design pairs contrast cases
Useful conceptual synthesis
Honest scope statements
Weaknesses
No uncertainty quantification on the core numbers
Small, English-only, non-random dataset sample
Operationalization gap between title and measurement
Audit method lacks protocol and reliability checks
Figures 1a/1b carry no analysis
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This Interspeech-format paper (§1, §5) audits queer (LGBTQIA+) inclusion in speech datasets and proposes a taxonomy of collection-practice tensions. The operationalization is deliberately narrow: queerness is measured only through gender identity annotations (transgender/non-binary vs. binary labels), because gender is the only queer-relevant field commonly available (§1, explicitly acknowledged). The central empirical claims are (1) measurable queer representation of 0–1.4% of speakers across six English speech technology datasets (Abstract, Table 1), (2) Common Voice 22.0's zero open trans/non-binary identification among 94,911 contributors (§3), and (3) Fairspeech's documented exclusion of non-binary speakers from its release ([35], p. 2). The conceptual claim is a four-tension taxonomy (§4): scaling vs. inclusion, efficiency vs. engagement, access vs. autonomy, static categories vs. fluid identities.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
How many queer speakers are in mainstream speech datasets?
According to this audit of six English speech technology datasets, measurable queer representation ranges from 0 to 1.4% of speakers: Common Voice 22.0 had 0 of 94,911 contributors identifying as transgender or non-binary, Casual Conversations V2 about 1.44%, and Edinburgh Accents 0.82% — levels the authors say are too low for robust bias measurement.
Why are there so few LGBTQIA+ voices in speech AI datasets?
The paper attributes this to practical tensions: open crowdsourced scaling builds no trust with stigmatized communities, pre-set mutually exclusive gender categories discourage participation, open-access release risks re-identification and voice misuse, and AI projects rarely disclose ethics review or engage communities before collecting sensitive identity data.
Which speech datasets did the paper audit?
Six technology datasets — Casual Conversations V2, Common Voice 22.0, Edinburgh Accents, English Dialects, Fairspeech, and L2-ARCTIC — plus two queer-specific corpora from the speech sciences, MAGES (gender-expansive speakers) and PTMV (transmasculine voices).
What are the four tensions in the paper's taxonomy?
Scaling versus inclusion (big open calls still miss stigmatized groups), efficiency versus engagement (fast crowdsourcing versus slow trust-building), access versus autonomy (open release versus community control over one's voice), and static categories versus fluid identities (fixed mutually exclusive gender labels versus free self-description).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1a/1b (CCV2 and EdAcc gender distributions) are bar charts whose numeric values are not extractable from the text, so the small-N structure of those two datasets could not be verified.
- Table 1 values reflect the authors' reading of external dataset documentation; this review did not independently verify each dataset's license, ethics disclosure, or label options.
- The extracted text appears complete (full body plus references), so truncation risk is low, but supplementary materials or datasheets, if any, were not part of the extraction.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.