North Small Translate: Advanced Cost-Effective Translation (Cohere CAT+)
Tom Kocmi, Alexandre Bérard, Phil Blunsom, Samuel Cahyawijaya, Shaun Cassini, Nicholas Frosst, Ona de Gibert, Aidan Gomez, et al.
cs.CL
Quick verdict
Cohere claims top <1T MT across 50 languages via difficulty-sampled SFT/DPO/RL; strong, honest eval but self-judged baselines and no significance tests.
Summary
Cohere's North Small Translate (entered into WMT26 as Cohere CAT+) is an open-weight, instruction-following machine translation model built on Command A+, a mixture-of-experts with 218B total and 25B active parameters, deliberately trained without test-time reasoning to keep production throughput high. Training uses difficulty sampling — retaining only documents where an early checkpoint still makes major errors — followed by five steps: coarse SFT (extending coverage from 35 to 50 languages, four merged models), fine-grained SFT on synthetic data, DPO, online RL with an LLM-judge reward, and a final light DPO that repairs RL regressions. Preference data comes from a draft-judge-post-edit loop that approximates on-policy pairs. On the WMT26 test set, released only after the model was finished, it scores 83.6 under a reference-free GEMBA-ESA judge (GPT-5.6), or 84.4 in Agentic mode, above Mistral Large 3 and Qwen 3.5-397B at 81.6; terminology, long-context, structured-translation and per-language results are also reported, and internal WMT25 progression rises from 71.3% to 81.8%. Main limitations: headline comparisons are self-judged under the authors' own protocol rather than official WMT26 human rankings; no significance testing; internal judges and training data are undisclosed; and the RL step brings no aggregate gain.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- North Small Translate achieves top MT quality among sub-1T-parameter systems across 50 languages on WMT26, scoring 83.6 GEMBA-ESA (84.4 with Agentic translation) versus 81.6 for Mistral Large 3 and Qwen 3.5-397B (Tables 1 and 4).
- Difficulty sampling plus a five-step SFT/DPO/online-RL/light-DPO protocol lifts internal WMT25 GEMBA-ESA from 71.3% to 81.8% (Table 2).
- A non-reasoning model prioritizing throughput matches the quality typically associated with reasoning models, with optional Agentic Translation recovering most of the remaining gap.
- WMT24-style test sets are near saturation: the same reference system commits major errors on only 15.8% of WMT24 segments versus 24.0% on WMT26 (Table 5).
Strengths
Contamination-aware evaluation design
Headline results use the WMT26 test set released only after model completion (§3.2), and §3.6 quantifies benchmark headroom: the same reference system shows major errors in 15.8% of WMT24 segments versus 24.0% on WMT26 (Table 5).
Unusual transparency for an industry report
Conflict of interest is disclosed (footnote 1: first author organizes WMT General MT); the RL step's regression is reported plainly (Table 2: DPO 80.1 → RL 80.0); full judge prompts are released (Appendix A); all 50 per-language scores are given (Table 6); cases where competitors win are shown (DeepL 81.8 vs 80.6 on WMT24++; DeepL 56.5 vs 48.9 on long-context; Gemma 4 92.1 vs 89.7 on terminology).
Data engineering with clear ablation evidence
Metric choice is empirically grounded
Breadth of evaluated capabilities
Weaknesses
Headline rankings are self-administered
No statistical uncertainty anywhere
Reproducibility gaps beyond the judge prompt
Unexplained evaluation anomalies in Table 1
Efficiency claims lack measurements; RL adds nothing on aggregate
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This industrial system paper presents North Small Translate (NST), Cohere's entry (as 'Cohere CAT+') into the WMT26 General MT, Terminology, and Automated MT Evaluation shared tasks. NST is an open-weight, instruction-following translation model built on Command A+ (MoE, 218B total / 25B active parameters), deliberately trained *without* test-time reasoning to maximize production throughput, with an optional 'Agentic Translation' pipeline (translate → judge → post-edit, all via the same model) for latency-insensitive use. Central claims: (1) top MT quality among sub-1T systems across 50 languages on WMT26 (Tables 1/4: GEMBA-ESA 83.6 plain, 84.4 agentic, vs 81.6 for Mistral Large 3 and Qwen 3.5-397B); (2) a five-step protocol — coarse SFT (35→50 languages, four merged models), fine-grained SFT on synthetic data, DPO, online RL (GSPO + LLM-judge reward), light final DPO — driven by difficulty sampling (§2.3.1); (3) near-saturation of WMT24-style test sets (Table 5: 15.8% vs 24.0% major-error segment rates for the same reference system).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is North Small Translate?
An open-weight machine-translation model from Cohere, entered into the WMT 2026 shared task under the name Cohere CAT+. It is fine-tuned from Command A+, a mixture-of-experts LLM with 218B total and 25B active parameters, covers 50 languages, and is trained to translate quickly without chain-of-thought reasoning while also handling post-editing, quality estimation, terminology glossaries, and JSON/HTML 'structured translation'.
Does North Small Translate beat Google Translate and DeepL?
According to the paper's own LLM-judge scoring on the WMT26 test set, it reaches 83.6 versus 68.2 for Google Translate. DeepL NextGen is much closer at 81.4 and actually beats it on the WMT24++ benchmark (81.8 vs 80.6) and on long-context translation (56.5 vs 48.9). Important caveat: all scores were produced by the authors' own judge protocol, not official WMT human rankings.
What is difficulty sampling in machine translation training?
A data-selection technique: run an early version of the model over candidate documents and keep only those where it still makes major errors (or loses to the human reference). This concentrates training on genuinely hard text, because a model that already translates a document flawlessly provides little learning signal from it.
Is the model open source?
The paper calls the model open-weight and links the base model (Command A+) on Hugging Face, but the text provides no download link for North Small Translate itself, and the training data, internal judge models, and hyperparameters are not disclosed.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Table 1 extraction is lossy: the Nemotron 3 Ultra and Qwen 3.6 27B rows contain only three of five metric values, so those cells were treated as unreliable and not cited.
- Review covers the provided arXiv v1 text only; official WMT26 shared-task findings and external leaderboards were not consulted.
- No figures appeared in the extraction; the review relies on tables and prose only.
- Several competitor references are 2026 preprints or blog posts, so baseline characterizations could not be independently verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.