Xiaomi-CocktailASR-1 Technical Report
Yiru Zhang, Hang Su, Lichun Fan, Ying Zeng, Chang Liu, Yifeng Wang, Yuquan Liang, Tao Li, et al.
cs.SD · cs.CL · eess.AS
Quick verdict
Claims SOTA LLM-based target-speaker ASR with rejection and CoT; strong benchmark numbers, but proprietary 1M-hour data, no ablations or significance testing.
Summary
Xiaomi-CocktailASR-1 is an LLM-based end-to-end target-speaker ASR (TS-ASR) system aimed at the cocktail party problem: using a 1–4 second reference recording as a voiceprint prompt, it transcribes only the designated speaker from mixed speech, with no separation front-end. The architecture concatenates reference audio, a one-second silence delimiter, and the mixture, encodes them with a 0.6B-parameter Data2Vec2-derived encoder, and feeds aligned embeddings into a Qwen3-8B LLM. Training uses roughly one million hours — ~400k hours of multi-speaker mixtures, ~600k hours of single-speaker pairs, ~10k hours of negative samples, plus template-generated chain-of-thought data — across four stages ending with SFT and reinforcement learning. Reported results: TS-WER of 4.11%/12.29% on LibriMix 2/3mix, 2.90%/4.91% on LibriSpeechMix, 21.81% on AMI-SDM, 20.63% on AliMeeting-Far; 1.73% WER on LibriSpeech single-speaker with 0.36% false rejection; rejection rates of 79.59%/75.35%/68.54% on three negative sets. CoT mode yields only marginal gains (e.g., 4.11→3.87 on LibriMix 2mix). Main limitations: no statistical significance testing, no ablations isolating component contributions, proprietary data at a scale public groups cannot match, and no code or model release.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A single LLM-based end-to-end model handles single-speaker ASR, multi-speaker target-speaker ASR, and negative-sample rejection without mode switching or a separation front-end.
- The model achieves state-of-the-art TS-WER on synthetic (LibriMix 2mix 4.11%, LibriSpeechMix 2mix 2.90%) and real-world (AMI-SDM 21.81%, AliMeeting-Far 20.63%) multi-speaker benchmarks.
- Internalized negative rejection reaches RR 79.59%/75.35%/68.54% on LibriSpeech Neg/Aishell Neg/in-house Neg while keeping FRR ≤0.73% on positive sets.
- A CoT reasoning mode provides explicit intermediate steps and yields modest WER improvements (e.g., 4.11%→3.87% on LibriMix 2mix).
Strengths
One model covers three regimes without mode switching
Tables 1–3 show the same system winning multi-speaker benchmarks (4.11% LibriMix 2mix), staying near parity single-speaker (1.73% LibriSpeech, 8.89% AMI-IHM — best among compared), and rejecting negatives, addressing the over-suppression problem stated in §1.
Metric design separates rejection distortion from accuracy
§4.2 introduces Non-empty WER (recomputed excluding empty outputs) alongside FRR, so the 0.36% LibriSpeech FRR and its WER impact are auditable rather than folded into one number.
Baselines include dedicated TS-ASR systems, not just weak general models
Largest win is on real far-field Chinese meetings
Demonstrated RR/FRR balance, not just asserted
Weaknesses
Blanket SOTA claim contradicted by its own Table 1
Zero statistical reporting
No ablations for any load-bearing design choice
~1M hours of largely proprietary data; nothing released
CoT is oversold relative to its measured benefit
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is an industrial technical report (Xiaomi Xiaoai Plus ASR team) on CocktailASR-1, an end-to-end target-speaker ASR (TS-ASR) system built on an LLM. Input is a 1–4 s reference utterance, a one-second silence delimiter, and the mixed signal; output is the target speaker's words, or an empty string when the target is absent, with an optional chain-of-thought (CoT) mode. Four claims are made (§1, §5): a single model handles single-speaker, multi-speaker, and rejection without mode switching; state-of-the-art TS-WER on synthetic (LibriMix, LibriSpeechMix) and real (AMI-SDM, AliMeeting-Far) benchmarks; internalized negative rejection with low false-rejection; and CoT gains in accuracy and interpretability.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is target-speaker ASR and the cocktail party problem?
In multi-person audio, target-speaker ASR transcribes only one designated person, identified by a short sample of their voice (a reference clip), while ignoring everyone else. The cocktail party problem is the difficulty of doing this when multiple people speak simultaneously. This paper's model takes a 1–4 second voice sample plus the mixed audio and outputs only the target person's words, or an empty string if that person is not speaking.
How does Xiaomi-CocktailASR-1 compare to Whisper or Qwen3-ASR?
General ASR models like Whisper Large-v2 and Qwen3-ASR-1.7b are not built for overlapping multi-speaker input: in the paper's Table 1 they score 36–161% WER on mixed-speech benchmarks because they transcribe everything or lose track of speakers. CocktailASR-1 uses a reference voice clip to lock onto one speaker, reaching 4.11% on LibriMix 2mix. On ordinary single-speaker audio it remains competitive (1.73% on LibriSpeech versus 1.58–1.87% for the general models).
Is Xiaomi-CocktailASR-1 open source?
No. The paper does not mention any code, model weights, or evaluation prompt release. Its training corpus (~400k hours multi-speaker, ~600k hours single-speaker, ~10k hours negative samples) is largely proprietary Xiaomi data, so the reported numbers cannot currently be independently reproduced.
What languages does CocktailASR-1 support?
The paper evaluates English (LibriSpeech, LibriMix, LibriSpeechMix, AMI) and Chinese (AliMeeting, Aishell, WenetSpeech-meeting, CommonVoice-zh, in-house Chinese negatives). Cross-lingual rejection is tested (English reference against Chinese negatives), but no other languages are reported.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–4 are images; only their captions are present in the extraction, so figure-internal details (e.g., the CoT sample layout in Figure 3, the pipeline diagram in Figure 4) could not be audited.
- Odd words in some headings (e.g., 'senorial', 'Singer Speaker Datasets') appear in the extracted text; whether they are PDF-extraction artifacts or typos in the original cannot be determined from text alone.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.