Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning
Mehrnaz Mofakhami, Ananya Sahu, Alejandro R. Salamanca, Daniel D'souza, Alexandre Berard, Thomas Euyang, Marzieh Fadaee, Julia Kreutzer
cs.CL
Quick verdict
SFT data mixing gives a 3.35B model >93% in-language reasoning over 60 languages at small accuracy cost; solid ablations, but L2 rate needs a trained trigger.
Summary
This Cohere Labs paper tackles the English-centricity of reasoning language models by asking how SFT data composition can teach a model to reason in the user's language ('L2 reasoning'). Starting from the 3.35B multilingual Tiny Aya base (context extended to 32K), the authors translate filtered English reasoning traces into 44 languages (<5K samples per language, ~286K total) using command-a-translate and DeepSeek-V3, and mix them with ~1.7M English reasoning traces and ~4.9M multilingual non-reasoning instruction data in a dual-mode SFT. The resulting Tiny Aya L2-Thinker reaches 93.8-98.3% L2 reasoning rate on non-English prompts across six benchmarks, with accuracy within a few points of an English-only twin except on competition-level PolyMath (11.1 vs 18.6). Ablations show broader reasoning-language coverage transfers L2 behavior to held-out languages without interference; a 10-30% fraction of non-reasoning data is a cheap lever (MGSM unseen accuracy 49->67 and L2 rate 46->89 at 10%); and joint mixing beats sequential adaptation and weight merging, the latter collapsing language conditioning. Key limitations, acknowledged by the authors: supervision is machine-translated and unmeasured for reasoning fidelity, no human evaluation was performed, and headline L2 rates depend on prepending a training-seen 'think in the prompt language' instruction.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A 3.35B model trained with SFT data mixing alone achieves an L2 reasoning rate above 93% across roughly 60 languages on six benchmarks, with accuracy within a few points of an English-only twin except competition-level math (PolyMath 11.1 vs 18.6).
- Broadening reasoning-language coverage improves L2 transfer to unseen languages without interference (unseen L2 rate rises monotonically with coverage, e.g., 14->35->60 on MGSM), so the curse of multilinguality does not apply to reasoning behavior.
- A small fraction of multilingual non-reasoning data is a cheap transfer lever: moving from 0% to 10% NR lifts MGSM unseen-language accuracy from 49% to 67% and L2 rate from 46% to 89%, with a sweet spot around 20-30%.
- Joint data mixing dominates sequential adaptation and linear weight merging: merging collapses in-language reasoning (seen L2 rates 59/17/49) while sequential scatters the fallback language, whereas mixing fails predictably to English.
Strengths
Systematic, well-controlled data-composition ablations
§5.1 varies coverage (1-language specialists / 2-language regional / 10-language joint) with region-balanced averaging and pre-fixed seen/unseen splits (Table 4); §5.2 sweeps NR fraction at matched training steps; §5.3 sweeps English fraction from a multilingual-only start; Appendix G adds mixing vs sequential vs merging under fixed budget. Each factor is isolated cleanly.
Transparent accounting of the accuracy trade-off
§4.2 does not hide the cost: PolyMath drops 18.6 (En-Thinker) to 11.1 (L2-Thinker), and the Qwen gap is decomposed into 21.7 points (base capability) vs 7.5 (L2 switch), attributed to the missing RL stage. The En-Thinker twin (same base/data, English traces) is a clean internal control in Table 1.
Unusually broad evaluation across capabilities and resource tiers
Actionable negative results
Artifact release enabling follow-up work
Weaknesses
Headline metric depends on a training-seen trigger phrase
Translation fidelity of the core supervision is unquantified
Purely descriptive statistics
Cross-scale, cross-regime baseline comparisons
L2 rate says nothing about reasoning quality
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets the English-centricity of reasoning LLMs: models answer non-English prompts but reason in English, risking intent loss ('lost in translation') and uninspectable traces for non-English users. It defines *L2 reasoning* - chain-of-thought in the prompt's language - and asks how SFT data composition and scheduling can induce it at 3.35B scale. Three claims structure the analysis: (i) broader reasoning-language coverage transfers the behavior to unseen languages without interference (§5.1); (ii) a small fraction of multilingual non-reasoning (NR) data is a cheap transfer lever (§5.2); (iii) an English reasoning (ER) backbone is necessary for hard math, and joint mixing beats sequential adaptation and weight merging (§5.3, §G). The deliverable, Tiny Aya L2-Thinker, reportedly exceeds 93% L2 rate across ~60 languages on six benchmarks (abstract; Table 1).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is 'L2 reasoning' in this paper?
Reasoning in which a language model's chain-of-thought is written in the same language as the user's prompt, instead of defaulting to English. The paper trains Tiny Aya L2-Thinker (3.35B) to do this for 45 trained languages and shows the behavior transfers to additional languages never supervised.
Does making a model reason in the user's language hurt accuracy?
In this paper, switching a 3.35B model from English-only to in-language reasoning cost at most 2-3 accuracy points on five of six benchmarks; the exception was competition-level math (PolyMath), where accuracy fell from 18.6 to 11.1 for the same base model, which the authors attribute to the absence of an RL training stage.
Do you need reasoning training data in every language to get in-language reasoning?
No. The paper argues that a small amount of machine-translated reasoning data (under 5K samples per language across 44 languages), broad language coverage, an English reasoning backbone, and a 10-30% share of multilingual non-reasoning data suffice - the in-language reasoning behavior generalizes to languages that never received reasoning supervision.
Are the model and dataset from this paper publicly available?
Yes. The authors release model weights (tiny-aya-l2-thinker, tiny-aya-en-thinker, tiny-aya-base-32K) and the multilingual reasoning dataset (~286K translated traces across 44 languages) on Hugging Face under the CohereLabs organization.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with two marked '[...sampling gap...]' regions; middle content (parts of §5.3-§5.4, §6.1-§6.3, and per-language Tables 8-10) was skipped, and the excerpt ends mid-way through appendix Table 12 (Macaron-MCQ per-language results).
- Whether translation-quality metrics, significance tests, or judge-validation analyses exist inside the skipped regions cannot be verified from this excerpt; all absence claims in this review are scoped to the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.