Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation
Daxin Tan, Dehua Tao, Chengxi Deng, Hanlin Zhang, Xiao Chen
eess.AS · cs.CL · cs.SD
Quick verdict
Defines output-mode gap in speech LLMs and cuts it via self-distillation; strong two-model results, but spoken gains are smaller and untested statistically.
Summary
The paper identifies and quantifies an 'output-mode gap' (OMG) in speech LLMs that generate interleaved text and acoustic tokens: given identical speech input, the internal text produced in speech-to-text-and-speech (S2TS) mode is markedly less accurate than speech-to-text (S2T) output. On Step-Audio-2-mini, OMG reaches 42.87 pp on Spoken-MQA and 29.72 pp on speech-rendered GSM8K; Baichuan-Audio-Instruct shows smaller gaps (12.41/10.39 pp). The authors propose Joint-Output On-Policy Distillation (JO-OPD): the model's own stronger S2T policy supervises text predictions along student-generated S2TS trajectories, with the teacher conditioning on a text-only projection of the student's history, plus a preservation loss regularizing non-text predictions. After training on 27,847 Tulu 3/NaturalReasoning prompts with synthesized speech, JO-OPD-Soft cuts OMG to 16.26 pp (Spoken-MQA) and 13.04 pp (GSM8K) on Step-Audio-2 with S2T accuracy unchanged, outperforming matched Self-SFT and Response-SFT baselines; Baichuan gains are smaller. Ablations support soft targets, text-vocabulary restriction, and non-text preservation. However, ASR-based spoken-answer accuracy improves far less than internal text (+7.49 pp on Spoken-MQA for Step-Audio-2, with slight GSM8K spoken regression), and single samples per input, no significance testing, and no code release limit confidence.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Interleaved speech LLMs exhibit a large output-mode gap: given identical speech input, internal text in S2TS mode is substantially less accurate than S2T output (42.87 pp on Spoken-MQA for Step-Audio-2-mini, Table 1).
- JO-OPD, distilling the model's own S2T policy onto student-generated S2TS trajectories with text-only history projection, reduces OMG to 16.26/13.04 pp on Step-Audio-2 while leaving S2T accuracy unchanged.
- Matched SFT baselines fail to close the gap — Self-SFT worsens S2TS(T) (32.45→23.61 on Spoken-MQA) — showing on-policy supervision at joint-generation prefixes is the active ingredient.
- Text supervision restricted to the text vocabulary plus non-text preservation is necessary: full-vocabulary supervision collapses spoken-answer accuracy from 36.16% to 11.20% (Table 5).
- Internal-text improvements only partially transfer to spoken answers: ASR-based spoken accuracy on Spoken-MQA improves 7.49 pp for Step-Audio-2 (Table 4), with a slight GSM8K regression.
Strengths
Novel, precisely formulated problem
OMG is defined formally (Eq. 1) and cleanly separated from input-side modality-gap work (Sec. 2.2); first quantification on two open interleaved architectures, e.g., 42.87 pp on Spoken-MQA for Step-Audio-2-mini (Table 1).
Well-matched baselines make the OPD contrast meaningful
Self-SFT and Response-SFT share JO-OPD's prompts and optimization updates (Sec. 4.2); Self-SFT actually degrades S2TS(T) to 23.61% on Spoken-MQA (Table 1), isolating on-policy supervision at joint-history prefixes as the active ingredient.
Ablations cover every design decision
Evidence beyond synthetic speech
Transferable design insight
Weaknesses
Weak transfer to actual spoken answers
No statistical rigor
Narrow model coverage
Synthetic single-voice training speech
Reproducibility gaps and no external comparison
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper studies an output-side discrepancy in speech LLMs with fully discrete interleaved generation (Step-Audio 2, Baichuan-Audio): with the speech input and model fixed, internal text produced in speech-to-text-and-speech (S2TS) mode is far less accurate than speech-to-text (S2T) output — the "output-mode gap" (OMG, Eq. 1). Claims: (1) OMG is large — 42.87 pp on Spoken-MQA and 29.72 pp on speech-rendered GSM8K for Step-Audio-2-mini, 12.41/10.39 pp for Baichuan-Audio-Instruct (Sec. 1); (2) JO-OPD, a self-distillation from the model's own S2T policy onto student-generated S2TS trajectories with text-only history projection plus non-text preservation, cuts OMG to 16.26/13.04 pp on Step-Audio-2 and beats matched SFT baselines; (3) ablations attribute the gains to soft teacher targets, text-vocabulary restriction, and preservation. The claims are well-scoped and mostly supported by the tables, with the important caveat that spoken-side improvements are much smaller than internal-text improvements.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the output-mode gap (OMG) in speech language models?
It is the accuracy drop that appears when a speech LLM must also speak: given the same speech input, the hidden text generated during joint text-and-speech (S2TS) generation answers questions less accurately than the model's plain speech-to-text (S2T) output. In this paper the gap reaches 42.87 percentage points on Spoken-MQA for Step-Audio-2-mini, because generated acoustic tokens contaminate the context used for later text predictions.
What is JO-OPD and how does it reduce the gap?
Joint-Output On-Policy Distillation uses the model's own stronger S2T mode as a teacher. The student first generates its own interleaved text-plus-speech trajectories; during training, at every text position the teacher sees only the text projection of that history while the student sees the full interleaved history, and a preservation loss keeps speech-generation behavior unchanged. Training on about 28,000 prompts cut OMG from 42.87 to 16.26 points on Spoken-MQA for Step-Audio-2.
Does JO-OPD improve the actual spoken answers, not just the hidden text?
Only partially. On Spoken-MQA, ASR-transcribed spoken-answer accuracy rose 7.49 points for Step-Audio-2 (28.67%→36.16%) and about 2.9 points for Baichuan-Audio. On GSM8K, Step-Audio-2's spoken accuracy stayed roughly flat (36.16%→35.25%) while its hidden text improved by over 17 points, so the acoustic generation side remains a bottleneck.
Which models and data were used, and is code available?
Experiments use the open Step-Audio-2-mini and Baichuan-Audio-Instruct models, trained on 27,847 prompts from Tulu 3 and NaturalReasoning with speech synthesized by the flite 'SLT' voice, and evaluated on Spoken-MQA, speech-rendered GSM8K, and human-recorded VoiceBench-BBH questions. The paper reports hyperparameters but does not announce a code release.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 is available only as garbled embedded text; diagram-level details (control-token handling, exact wiring) were inferred from the prose of Sec. 3.
- Equations and Fig. 1 caption are partially mangled by extraction, but all headline numbers were cross-checked between abstract and Tables 1-6.
- Short-format (Interspeech-style) paper; implementation details may exist in unreleased code or future extensions.
- Several cited works are 2026 preprints that could not be verified offline.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.