Transcribe, Translate, and Optimize: Joint Reward Learning for Speech Translation
Yanghe Dong, Wanting Huang, Weiran Wang
cs.CL · eess.AS
Quick verdict
Proposes joint ASR-ST GRPO for CoT speech translation; beats SFT and Direct GRPO on CoVoST 2/FLEURS, but single seed, one model, no significance tests.
Summary
The paper tackles the training-inference mismatch in transcription-based chain-of-thought (CoT) speech translation: supervised fine-tuning (SFT) trains translation on reference-transcript prefixes, while inference conditions on model-generated transcripts. The authors jointly fine-tune recognition and translation with group relative policy optimization (GRPO, DAPO variant without KL), scoring the sampled transcript via 1-WER and the translation via chrF++ (or BLEU) scaled to [0,1], with a binary format gate; three token-advantage assignments (Fully Coupled, Asymmetric, Decoupled) are compared. Qwen2.5-Omni-3B is trained on 289k English-to-Arabic/Chinese/German/Japanese CoVoST 2 utterances and evaluated on CoVoST 2 and FLEURS. CoT GRPO attains the best average BLEU/chrF++ on both test sets: +1.77/+0.83 BLEU over Direct ST GRPO and +0.82/+0.67 over CoT SFT, with 8.8%/7.2% relative WER reductions. Fully Coupled advantages yield the highest validation BLEU, and a transcript-intervention experiment shows GRPO retains its translation edge when given ground-truth transcripts. Strengths: systematic validation ablations, honest reporting of mixed effects, and a genuinely informative intervention analysis. Limitations: one model and single runs with no significance tests or variance estimates; several ablation choices rest on sub-point differences; no empirical comparison with prior mismatch mitigations such as corrupted- or predicted-transcript training; no code release; the FLEURS test set is small (1.77 hours).
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- CoT GRPO with joint ASR-ST rewards outperforms Direct ST GRPO by 1.77 and 0.83 average BLEU on CoVoST 2 and FLEURS test sets.
- Compared to CoT SFT, GRPO improves average BLEU by 0.82 and 0.67 points while reducing WER by 8.8% and 7.2% relatively on the two test sets.
- An explicit ASR reward during joint training substantially improves transcription accuracy, with most of the WER reduction already present at λ=0.25.
- Fully Coupled token advantages yield the highest BLEU among the three assignment strategies on both validation sets under a shared configuration.
- GRPO retains a translation advantage over SFT when both receive ground-truth transcripts, indicating it better exploits transcript content for translation.
Strengths
Reward design removes the mismatch at its source
The translation reward is computed on translations conditioned on the model's own transcripts within the same rollout (Sec 3.2, Eq. 4-6), rather than on reference-prefix conditions, directly targeting the training-inference mismatch documented in [13]; the gated joint reward r_ST + λ·r_ASR is a clean formulation.
Systematic, sequential validation ablations before test
Table 1 sweeps reward metric (BLEU vs chrF++), reference-aware training and group size, and ASR weight λ on validation sets before fixing the test configuration; the λ sweep shows most WER reduction already at λ=0.25 (6.76→5.64 CoVoST 2), supporting the explicit ASR-reward claim.
Transcript-intervention analysis isolates the mechanism
Honest, precise reporting of negative and mixed results
Reproducible protocol details
Weaknesses
No statistical rigor: single runs, no significance testing
Missing direct baselines for the mismatch problem
Non-standard initialization: GRPO from base, not from SFT
Narrow scope and thin cross-corpus evaluation
Protocol asymmetry in WER measurement
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This Interspeech-format paper targets a specific failure mode in LLM-based speech translation: CoT systems that emit a transcript then a translation are fine-tuned (SFT) with reference-transcript prefixes, so at inference the translation conditions on error-prone model-generated transcripts - the training-inference mismatch documented by [13]. The remedy is reinforcement fine-tuning with GRPO in DAPO form (asymmetric clipping ε_low=0.2/ε_high=0.28, no KL), where each sampled response is scored on both segments: r_ASR = clip(1−WER) with Whisper normalization (Eq. 5), r_ST = sentence-level chrF++/100 (retained over BLEU reward in §5.1.1), and a binary format gate (Eq. 6). Three token-advantage assignments (Fully Coupled / Asymmetric / Decoupled) and optional reference-aware training are ablated. Using Qwen2.5-Omni-3B trained on CoVoST 2 en→{ar,zh,de,ja} (289,412 utterances, 430.12 h, no repeated source recordings), the claims are: (i) CoT GRPO beats Direct ST GRPO by +1.77/+0.83 average BLEU on CoVoST 2/FLEURS test; (ii) it beats CoT SFT by +0.82/+0.67 BLEU with 8.8%/7.2% relative WER cuts; (iii) the explicit ASR reward drives most WER gains; (iv) Fully Coupled advantages are best.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What problem does this paper solve?
In LLM speech-translation systems that first transcribe then translate (chain-of-thought), models are trained to translate after perfect reference transcripts but at inference must translate after their own error-prone transcripts. The paper closes this training-inference gap with reinforcement fine-tuning (GRPO) that scores both the transcript and the translation of the model's own outputs during training.
How much does GRPO improve over standard supervised fine-tuning here?
On CoVoST 2 and FLEURS test sets with Qwen2.5-Omni-3B (English to Arabic, Chinese, German, Japanese), CoT GRPO beats CoT SFT by 0.82 and 0.67 average BLEU and cuts transcription WER by 8.8% and 7.2% in relative terms. It also beats direct-translation GRPO by 1.77 and 0.83 average BLEU.
What is the Fully Coupled advantage strategy?
One of three ways to assign the group-relative advantage to output tokens. Under Fully Coupled, every token in the response (both transcript and translation parts) receives the advantage of the joint reward (translation score plus λ times the ASR score). The alternatives give transcript tokens the ASR-only advantage (Decoupled) or the joint one (Asymmetric). Fully Coupled produced the best BLEU in validation.
Are the code and datasets available?
The paper uses public datasets (CoVoST 2 and FLEURS) and the open ms-swift training framework, and it gives full prompts and hyperparameters, but no code repository or model checkpoints are mentioned in the text.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Text extraction appears complete including all four tables; the paper contains no figures, so no visual material could be analyzed.
- Training curves, reward statistics, and format-gate rates are absent from the text, so GRPO training stability and gate behavior cannot be assessed.
- The single-column venue format leaves some SFT details (epochs, loss masking over tags) unstated; these were not guessed in this review.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.