KABURI-TTS: Phoneme-Keyed Activity-conditioned Bi-channel Utterance Rendering for Interaction
Ryuichiro Higashinaka, Shinnosuke Takamichi, Tetsuji Ogawa
cs.SD · cs.CL
Quick verdict
KABURI-TTS conditions stereo dialogue TTS on a phoneme/activity raster; human evals show better interaction naturalness, but baseline coverage is thin.
Summary
KABURI-TTS is a Japanese two-channel spoken-dialogue TTS that takes per-speaker phoneme rasters as input, derives voice activity and a ten-class interaction state from them, and renders both speakers on separate stereo channels using DACVAE latents with a rectified-flow DiT backbone (Irodori TTS) adapted via LoRA. It is trained on LLM-jp-Zoom1 (1,085 hours, 60 speakers) segmented into 30-second chunks, with auxiliary BCE heads for activity, onset/offset, overlap, and turn-switch prediction plus L2-SP regularization. Two raster frontends are proposed: corpus-statistics placement and a 0.8M-parameter duration predictor. In crowdworker evaluations (140 workers, two raters per sample; Wilcoxon signed-rank with Benjamini-Hochberg FDR), KABURI (stat) achieved the best text-input results, with interaction naturalness 3.83 significantly above Irodori (stat) and KABURI (pred), and no significant difference against MOSS-TTSD (corrected p=0.071 tendency); in Test chunk Eval, every system stayed below ground truth (utterance naturalness 4.42). Voice-activity analysis shows overlap and turn-switch rates closer to real dialogue than the Irodori baselines. Main limitations: no head-to-head comparison with the stereo-capable systems cited in Related Work (CHATS, CoVoMix, Moshi multistream, ZipVoice-Dialogue-Stereo), the MOSS-TTSD baseline is monaural, only two raters per sample, frame-level adherence of output to the input raster is unverified, and the learned predictor's interaction naturalness (3.30) trails statistical placement.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Conditioning a two-channel TTS on an explicit per-speaker phoneme raster, with activity and a ten-class interaction state derived from it, enables controllable one-speaker-per-channel dialogue synthesis with reliable simultaneous speech.
- In human evaluations, KABURI-TTS attains higher utterance- and interaction-level naturalness than the Irodori-based baselines, with KABURI (stat) best overall in Text-in Eval (interaction naturalness 3.83).
- The method produces more overlap and more frequent turn-taking than baselines, close to ground-truth statistics (overlap 0.105 and 35.6 switches/min for KABURI (stat) vs GT 0.076 and 35.4 in Test chunk Eval).
- The raster frontend matters: corpus-statistical placement beats the trained 0.8M-parameter duration predictor on interaction naturalness in both evaluations.
Strengths
Explicit conditioning turns overlap into a controllable input
Unlike text-serialized systems (SoundStorm, MOSS-TTSD; Sec. II), KABURI-TTS conditions on a per-speaker phoneme raster from which the activity raster (a=1[p≠sil]) and a ten-class interaction state are derived (Sec. III-B); Table III shows resulting overlap/turn statistics near GT (0.105 vs 0.076 overlap; 35.6 vs 35.4 switches/min).
Well-designed ablation ladder
Sec. IV-B separates acoustic-model from placement effects: KABURI with GT/stat/pred rasters vs Irodori with GT/stat placement isolates exactly where quality is lost, with the shared statistical-placement frontend keeping comparisons controlled.
Statistical protocol above venue norm
Candid self-limitation
Release and detail level support reuse
Weaknesses
No stereo baseline actually evaluated
Thin, uncalibrated human ratings
Learned placement underperforms
Controllability only indirectly evidenced
Fixed 30-second windows and an unexplained inversion
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
KABURI-TTS (APSIPA ASC 2026, arXiv:2609.07200) targets a concrete gap: two-channel, one-speaker-per-channel dialogue TTS for training full-duplex spoken-dialogue models, where prior text-driven systems (SoundStorm, MOSS-TTSD, CoVoMix, CHATS, Moshi-multistream, ZipVoice-Dialogue-Stereo; Sec. II) learn overlap implicitly and cannot control it. The central design claim is that an explicit per-speaker phoneme raster — from which the activity raster (a=1[p≠sil]) and a ten-class interaction state (4 base + 6 boundary classes, Sec. III-B) are derived — yields controllable, human-like simultaneous speech. Three claims are tested: (1) higher utterance- and interaction-level naturalness than baselines; (2) voice-activity statistics closer to real dialogue; (3) controllability via interchangeable raster frontends. Code and pretrained models are released (github.com/llm-jp/kaburi-tts).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is KABURI-TTS?
KABURI-TTS is a Japanese text-to-speech model for two-party spoken dialogue. It takes a per-speaker phoneme timeline (a phoneme raster) plus voice-activity information and renders each speaker's voice on a separate stereo channel, explicitly controlling backchannels, interruptions, and overlapped speech. 'Kaburi' is a Japanese word meaning simultaneous speech.
What data was KABURI-TTS trained on?
It was trained on LLM-jp-Zoom1, a large-scale Japanese casual-conversation corpus of about 2,000 Zoom conversations (1,085 hours, 60 speakers) split into 30-second chunks; transcriptions and phoneme alignments were produced with Silero VAD, Whisper, and Montreal Forced Aligner.
Is the KABURI-TTS code and model available?
Yes. The paper states that code and pretrained models are released at github.com/llm-jp/kaburi-tts. The backbone (Irodori TTS) and the LLM-jp-Zoom1 training corpus are also publicly available.
How does KABURI-TTS differ from MOSS-TTSD or SoundStorm?
Those models generate a single mixed monaural track from serialized speaker-tagged text, so overlap is learned implicitly and is hard to control. KABURI-TTS outputs two separate channels conditioned explicitly on a phoneme/activity raster; in the paper's Text-in evaluation KABURI (stat) scored higher interaction naturalness (3.83 vs 3.53), though the difference versus MOSS-TTSD was not statistically significant (corrected p=0.071).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Table I/II significance superscripts are flattened by PDF extraction (e.g., 'ddeef f'); pairwise significance mappings were reconstructed from the flattened notation and carry minor ambiguity.
- Fig. 1 could not be inspected visually; the architecture description was cross-read from the Sec. III prose, which is detailed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.