ReMova: Fine-tuning LLMs for English to Belarusian translation
Mikita Pilinka, Aliaksandr Kliujeŭ, David Samuel, Yves Scherrer
cs.CL
Quick verdict
ReMova adds Belarusian-specific cleaning to LLM fine-tuning; matched ablation supports data-quality as key bottleneck, though gains lack significance tests.
Summary
The paper introduces REMOVA, a data-cleaning pipeline tailored to Belarusian, and uses its output to fine-tune English→Belarusian translation models from two paradigms: NLLB-200-3.3B with full fine-tuning, and Gemma-4-12B-IT and TranslateGemma-12B-IT with LoRA. The pipeline combines ftfy, NFKC normalisation, GlotLID language identification, the Belarusian-Wikipedia Wikificator, LanguageTool spell-checking, MinHash-LSH deduplication and per-corpus BLASER thresholds, targeting the dual-orthography problem (official Narkamaŭka vs Taraškievica), Russian/Ukrainian/Polish interference and machine-translated web noise. Training uses roughly 169,000 pairs from HPLT, OpenSubtitles, Tatoeba and Wikimedia, plus 4,600 back-translated pairs from curated spoken-register HPLT segments. On the BOUQuET test set (854 sentences, eight domains), the fine-tuned TranslateGemma leads with 32.63 BLEU, 55.36 chrF++ and 90.03 COMET, narrowly ahead of Gemma-4 (31.40 BLEU) and NLLB (31.22). A matched ablation, re-training each model on unfiltered data of identical volume and composition, shows filtering gains of +1.99 BLEU for NLLB versus +3.62 (Gemma-4) and +4.38 (TranslateGemma) — roughly twice as much for the LLMs — supporting the authors' data-quality-bottleneck thesis. Main limitations, largely acknowledged: no human evaluation, no statistical significance testing, COMET not meta-evaluated on Belarusian, no commercial-system baselines, and a confound between model architecture and fine-tuning regime (full FT vs LoRA).
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The REMOVA pipeline (Wikificator + LanguageTool + GlotLID/BLASER/dedup stages) effectively removes dual-orthography variants, cross-language interference and misspellings from Belarusian parallel data.
- Fine-tuning on REMOVA-cleaned data outperforms fine-tuning on volume- and composition-matched unfiltered data for all three models tested.
- Decoder-only LLMs gain roughly twice as much from filtering as the encoder-decoder NLLB (+3.62 BLEU for Gemma-4 and +4.38 for TranslateGemma vs +1.99 for NLLB), suggesting data quality is a first-order bottleneck specific to how LLMs absorb noisy low-resource data.
- The fine-tuned TranslateGemma-12B-IT is the best reported open EN→Belarusian system here, reaching 32.63 BLEU / 55.36 chrF++ / 90.03 COMET on BOUQuET and leading on every reported aggregate metric.
Strengths
Language-specific cleaning targets a documented, real problem
§1–2 ground the pipeline in the dual-orthography situation (Narkamaŭka vs Taraškievica), Russian/Ukrainian/Polish interference, and MT-contaminated web text (Koshchanka & Martysiuk 2025), then operationalise it with community tools: Belarusian-Wikipedia Wikificator regexes plus LanguageTool backed by the National Corpus (§2.1).
Matched ablation design isolates the filtering effect
§3.3 re-trains all three models on unfiltered data matched in total volume, per-source proportions, hyperparameters and eval protocol, so Table 5's base/unfilt./+FT deltas are attributable to cleaning rather than data quantity — an unusually clean control for this literature.
Full stage-level transparency of the pipeline
Cross-paradigm results with per-domain granularity
Candid limitations section and released artefacts
Weaknesses
No statistical significance testing for the central claim
Architecture claim confounded with fine-tuning regime
Base-model evaluation protocol inconsistency
Pipeline thresholds lack sensitivity analysis
No commercial or community-system calibration
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ReMova (arXiv 2609.16427, cs.CL) addresses English→Belarusian machine translation with two coupled contributions. First, REMOVA, a Belarusian-specific parallel-data cleaning pipeline motivated by an unusually rich sociolinguistic framing (§1): the coexistence of two normative orthographies (official Narkamaŭka vs Taraškievica), interference from Russian/Ukrainian/Polish amplified by 'new speakers', and pervasive unreviewed machine translation whose outputs (dating back to the Belazar system, per Koshchanka & Martysiuk 2025) have entered ordinary usage. Second, fine-tuned translation models targeting the WMT26 constrained track (<20B params, non-commercial licence): NLLB-200-3.3B with full fine-tuning, and Gemma-4-12B-IT plus TranslateGemma-12B-IT with LoRA. The central claims are that (a) filtering improves all three fine-tuned models over matched unfiltered training, (b) the decoder-only LLMs gain roughly twice as much (+3.62/+4.38 BLEU) as the encoder-decoder NLLB (+1.99), and (c) data quality is therefore a first-order bottleneck for Belarusian MT.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ReMova and what does it achieve?
ReMova is a two-part contribution from University of Oslo researchers: (1) REMOVA, a data-cleaning pipeline specialised for Belarusian that fixes dual-orthography variants, misspellings and interference from Russian-like languages in parallel corpora, and (2) fine-tuned English→Belarusian translation models. Its best model, a LoRA-tuned TranslateGemma-12B-IT, reaches 32.63 BLEU, 55.36 chrF++ and 90.03 COMET on the BOUQuET test set (854 sentences), ahead of fine-tuned Gemma-4-12B (31.40 BLEU) and fully fine-tuned NLLB-3.3B (31.22 BLEU).
Why is Belarusian machine-translation data so noisy?
The paper documents three overlapping causes: Belarusian has two competing normative systems (official Narkamaŭka and Taraškievica) that mix within single texts; interference from Russian, Ukrainian and Polish fills gaps left by scarce Belarusian web data, amplified by many 'new speakers' raised in Russian-speaking families; and a large share of online Belarusian text is unreviewed machine translation whose artificial forms have been absorbed into ordinary usage.
How much does data filtering improve translation quality in this study?
In a matched ablation where each model was re-trained on unfiltered data of identical volume and source composition, moving to the REMOVA-cleaned corpus improved BLEU by +1.99 for NLLB-3.3B, +3.62 for Gemma-4-12B and +4.38 for TranslateGemma-12B — roughly twice as much for the LLM-based models as for the dedicated encoder-decoder MT system. chrF++ and COMET moved in the same direction, though no statistical significance tests were performed.
Is the ReMova model and code publicly available?
Yes. The paper links a released model on Hugging Face (mineralsfree/ReMova-TranslateGemma-en-be) under the Gemma Terms of Use, and a GitHub repository (mineralsfree/ReMova) for the cleaning pipeline. Training hyperparameters are published, though some pipeline details such as the exact curated LanguageTool dictionary edits and per-corpus BLASER thresholds may need to be checked in the repo.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 (pipeline schematic) is available only as flattened extraction text; stage connectivity was reconstructed from §2 prose and the caption.
- Release completeness of the curated LanguageTool dictionary edits and per-corpus BLASER threshold values could not be verified from the paper text alone.
- The appendix tokenizer study (Table 6) is included in the text but only tangentially related to the main claims; it was reviewed but not weighted in the verdict.
- Some cited works (e.g., Gemma 4, TranslateGemma, WMT26 task page) are newer than this reviewer's verification horizon; their characterisation is taken from the paper's own descriptions.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.