SalamandraTA at WMT 2026 Terminology Shared Task: Hard Examples Are Better Teachers
cs.CL
Quick verdict
硬样本过滤在固定数据量下将术语准确率从 78.7% 提升至 89.9%；控制变量消融实验扎实，但相较于随机采样的优势并不显著（+1.3 个百分点）。
Summary
本文介绍了巴塞罗那超级计算中心（BSC）参加 WMT26 术语共享任务赛道 1（Terminology Shared Task Track 1）的提交结果，以及其背后的数据筛选思路。在基于词汇表标注的微调数据中，大多数样本要求的术语模型本身就会生成；作者仅保留了参考模型 SalamandraTA-7B-instruct v2.0 在无术语提示翻译时遗漏了至少一个规定术语的样本。在一项固定 76,000 个实例的 EMEA 衍生术语组件和固定 v2 背景混合（682,431 个实例）的对照研究中，将硬样本比例从 0% 提高到 100%，使得 WMT25 测试集（en→de/es/ru，共 1,589 个术语）上的术语准确率从 78.7% 提升至 89.9%（提升 11.2 个百分点，95% 置信区间 [+9.5,+12.8]，p<0.0001）；与从未过滤池中随机抽样相比（该池天然富含硬样本），过滤仅额外带来 1.3 个百分点的提升（p=0.003）。术语数据（包含 94 个语言对、29 种语言的 33,615 个实例）来自基于 Gemma-4-31B 并经过 LaBSE 过滤的两个合成流水线，并在 SalamandraTA-7B-instruct v3.0 中发布。在 WMT26 官方评估中，未经修改的模型结合分块推理（chunk-level inference）、分块词汇表以及受术语准确率保护的 CometKiwi 后编辑，取得了 94.2% 的术语成功率和 74.6 的 chrF++；在 22 个提交中，仅有 COZY 和 Agentic-OPUS 在两项指标上均优于它。在 WMT25 上，它仅通过 SFT 就超越了作者之前的 GRPO 系统（94.0 对比 91.3 的术语准确率），尽管该比较混合了不同的指令微调数据集。局限性：医疗领域偏差、绑定 v2.0 的静态过滤器，以及狭窄的消融实验范围（单一数据源、三个测试方向、单一尺度）。
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Filtering glossary-annotated SFT data to examples where reference model v2.0 mistranslates at least one prescribed term raises term accuracy from 78.7% to 89.9% at fixed data volume (+11.2 points, 95% CI [+9.5,+12.8], p<0.0001, §4.2).
- Against a random sample of the same unfiltered pool — naturally ~52.5% hard — the filter still adds +1.3 points (p=0.003, §4.2).
- The public SalamandraTA-7B-instruct v3.0, unmodified and wrapped in chunked inference with per-chunk glossaries and QE-guarded post-editing, achieves 94.2% term success at 74.6 chrF++ in WMT26 Track 1, with only COZY and Agentic-OPUS beating it on both metrics (Table 1).
- On WMT25, v3.0 surpasses the authors' prior GRPO system with SFT alone (94.0 vs 91.3 term accuracy; 69.4 vs 67.3 chrF, Table 2), though the authors acknowledge this comparison is not fully controlled.
Strengths
Genuinely controlled ablation design
Section 4.1 fixes the v2 background mixture (682,431 instances, Table 5), identical hyperparameters (1 epoch, lr 1e-5 cosine, 3% warmup, 8,192 context, batch 32), and an EMEA-only terminology component at 4,000/direction × 19 directions; only the hard-share varies across 0/25/50/75/100% plus random. Easy and hard pools pass the same cleaning, quality, and LaBSE gates, so difficulty is the only manipulated variable. Full per-direction numbers in Table 4.
Honest scoping of what is and is not controlled
Section 4.1 explicitly states the ablation runs do not reproduce the v3.0 recipe (v2 background, EMEA-only, fixed 76,000 instances); Section 6.2 flags the WMT25 GRPO comparison as system-level, not an ablation, because the instruction mixtures differ. This candor is above norm for shared-task papers.
Statistical hygiene
Open artifacts and detailed appendices
Insightful QE-vs-adherence analysis
Weaknesses
Headline framing inflates the practical effect
Self-referential, static hardness definition
Narrow ablation scope
Per-direction non-monotonicity undercuts the dose-response story
Slight quality cost at full-hard in the closest direction
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a WMT26 Terminology Shared Task Track 1 system description (BSC) built around one data-centric claim: glossary-annotated SFT data is largely redundant, and keeping only examples where a fixed reference model (SalamandraTA-7B-instruct v2.0, translating without terminology cues) misses at least one prescribed term concentrates the useful signal (§1, Figure 1). Three claims are advanced: (1) at fixed data volume this filter lifts term accuracy from 78.7% to 89.9% (+11.2 points, 95% CI [+9.5,+12.8], p<0.0001; §4.2, Table 4); (2) a two-way synthetic pipeline on Gemma-4-31B — MeSpEn terms→text (12,479 instances) and EMEA text→terms→documents (21,136 instances), 33,615 total over 94 directed pairs and 29 languages (§3, Appendix A) — trains the terminology component of the public v3.0 release; (3) the unmodified v3.0 wrapped in chunked inference with per-chunk glossaries and CometKiwi post-editing guarded by term accuracy reaches 94.2% term success at 74.6 chrF++, with only COZY and Agentic-OPUS better on both axes out of 22 submissions (Table 1), and surpasses the authors' prior GRPO system on WMT25 (94.0 vs 91.3 term accuracy; Table 2).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'hard examples are better teachers' mean in this paper?
When fine-tuning a translation model to follow glossaries, most training examples ask for terms the model would already produce without help. The authors keep only examples where the reference model's own unconstrained translation misses at least one prescribed term, and show in a controlled, fixed-volume study that this selection raises terminology accuracy.
How much did hard-example filtering improve terminology accuracy?
Going from 0% to 100% hard examples raised term accuracy on the WMT25 test set (en→de/es/ru) from 78.7% to 89.9% (+11.2 points, 95% CI [+9.5,+12.8]). However, against a random sample of the unfiltered pool — which is already fairly hard (~52.5%) — the filter added only +1.3 points (p=0.003).
How did SalamandraTA rank in the WMT26 terminology task?
The system scored 94.2% term success with 74.6 document-level chrF++ on Track 1. Of 22 submissions, only COZY and Agentic-OPUS scored better on both metrics; term success was 2.2 points below the human reference (96.4%).
Are the model and training data available?
Yes. SalamandraTA-7B-instruct v3.0 is on Hugging Face (BSC-LT), and the terminology data built by the two-way pipeline (33,615 instances, 94 language pairs) is part of its released instruction-tuning mixture. The pipeline itself uses open components: Gemma-4-31B for generation/extraction, LaBSE for filtering, Stanza for lemmatization, CometKiwi for quality estimation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–4 are present only as captions in the extracted text; Figure 2's plotted values were cross-verified against the numeric per-mixture results in Appendix D (Table 4), so no quantitative claim in this review depends on the images.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.