ConversationalVoice: Full-Duplex Speech Data from Real Conversations through Source-Faithful Reconstruction and Conversation-Grounded Expansion
Richard Yucheng He, Baodong Cao, Chen Xu, Yihang Liu, Tairan Chen
cs.CL · cs.SD · eess.AS
Quick verdict
A three-stage pipeline turns real dialogues into full-duplex training data; thoughtful design, but evaluation is automatic-only and downstream value untested.
Summary
The paper presents ConversationalVoice, a three-stage pipeline from AveraLabs that converts real two-speaker recordings into full-duplex speech training data. Stage one, quality-verified separation, uses diarization-based window planning, DialogueSidon joint separation/restoration, stable speaker-slot assignment, WavLM-based identity checks, and ASR to produce speaker tracks plus a canonical transcript. Stage two, source-faithful reconstruction, resynthesizes each utterance with a voice-cloning TTS using a composite reference (fixed speaker sample plus the separated utterance), then rebuilds turn order, pauses, and overlaps around synthesized durations, adding word alignment, audio tags, and delivery instructions. Stage three, expansion, generates new grounded continuations with an explicit schema for backchannels, paralinguistic events, and overlap placement. All stages share one artifact schema with equal-duration speaker tracks. Evaluation is entirely automatic: NISQA MOS rises from 3.563 (separation) to 4.405 (reconstruction) and 4.608 (expansion); WavLM same-speaker similarity stays at 0.983-0.991 with positive discrimination margins. Reconstruction fidelity against the source shows duration ratio 1.263 with turn/overlap/backchannel F1 of 0.693/0.664/0.860 (Table 3). Expansion rates run close to reconstruction (turns -4.6%, overlap events -8.0%, backchannels -13.2%, interruptions -16.0%), and a Gemini judge scores coherence 4.94/5 and naturalness 4.80/5. Main limitations: no dataset statistics (hours, languages, counts, or evaluated-pair N are never reported), no human listening tests, a single LLM judge, unnamed TTS and ASR components, and no downstream training validation.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Real two-speaker recordings can be converted into three complementary full-duplex training artifacts (separated, reconstructed, expanded) sharing one schema with word timing, audio tags, and delivery instructions.
- Quality-verified separation with stable speaker-slot assignment provides a validated substrate for downstream generation (same-speaker similarity 0.983, discrimination margin 0.200).
- Source-faithful reconstruction yields cleaner speech (NISQA MOS 3.563 to 4.405) while largely preserving interaction structure (backchannel F1 0.860; turn F1 0.693; overlap F1 0.664).
- Conversation-grounded expansion produces continuations judged coherent (4.94/5) and natural (4.80/5) with interaction rates close to reconstruction.
- The pipeline's value for full-duplex model training is asserted but explicitly not demonstrated, being deferred to future work.
Strengths
Stable speaker-slot assignment as a first-class design goal
Section 3.2 maps DialogueSidon's anonymous output slots to diarization speakers via activity correspondence and embeddings, then freezes the mapping, preventing a local permutation from becoming an identity switch; WavLM checks reject silent, leaking, merged, or split tracks before synthesis — a concrete advance over DuplexChat's endpoint (Ref. [6]).
Careful, position-preserving annotation contract
Audio tags are zero-duration anchors on the word timeline, plain text is derived from tagged text by the worker rather than generated independently, and a validation layer rejects substitutions or stage directions leaking into lexical content (Sec. 3.3-3.4, Table 1) — making the artifact directly usable for lexical-only, tagged, or instruction-conditioned objectives.
Documented engineering insight: align before assembly
Unusually candid limitations section
Sensible fidelity protocol for duration-changed outputs
Weaknesses
No dataset or evaluation statistics anywhere
Fully automatic evaluation with a single, unvalidated LLM judge
Circular references for WER and speaker identity
Modest fidelity framed generously; no baseline comparison
Reproducibility undermined by unnamed components
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
ConversationalVoice — Preliminary Review
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ConversationalVoice?
A data pipeline from AveraLabs that converts real two-speaker conversation recordings into three kinds of full-duplex speech training data: quality-verified separated speaker tracks, resynthesized reconstructions with word-level alignment and delivery instructions, and new AI-generated dialogue continuations grounded in the original conversation, speakers, and interaction pattern.
Does the generated data improve full-duplex speech models?
Not shown. The paper evaluates only data properties (speech quality, speaker similarity, interaction statistics, LLM-judged coherence and naturalness). Downstream training gains are explicitly left as future work, and the authors state the results do not establish downstream effectiveness.
How good is the reconstructed and expanded speech quality?
By automatic metrics: NISQA MOS is 4.41 for reconstruction and 4.61 for expansion, versus 3.56 for the separated source; WavLM same-speaker similarity stays at 0.983-0.991 with positive speaker-discrimination margins. A Gemini-based judge rated expansion 4.94/5 for coherence and 4.80/5 for naturalness. No human listening tests were performed.
Is the dataset or code available?
The paper lists a GitHub repository (github.com/avera-labs/ConversationalVoice), but it never states dataset size, languages, or licensing, and does not commit to releasing audio; its ethics section discusses what a responsible release should include (provenance, removal process, synthetic labeling) rather than announcing one.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 appears only as a caption in the extracted text; the diagram's visual content could not be reviewed.
- The extraction appears complete (text ends with reference [12] and the page-8 footer; all sections and tables 1-6 are present), and the GitHub repository's contents were not inspected - availability reflects the paper's stated URL only.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.