CantoneseLLM v2: Reasoning in a Low-Resource Language
Tsz Chung Cheng, Chung Shing Cheng, Chaak Ming Lau, Cheuk Hei Chong
cs.CL
Quick verdict
A transparent five-stage recipe for Cantonese reasoning LLMs with strong artifact release; evidence limited by single-run training and MCQ-heavy evaluation.
Summary
The paper documents the construction of CantoneseLLM v2, 8B and 30B-A3B reasoning models adapted from Qwen3 through five stages: continual pre-training on 784M tokens (5.4% Cantonese), training-free chat-vector merging for instruction following, SFT on a 74,865-row mixture, DPO, and RLVR with a multiplicative language-and-script reward. The central findings are diagnostic: chat-vector merging preserves the donor's non-Cantonese reasoning language; SFT with scarce Cantonese reasoning data collapses reasoning (8B: 64.5% empty reasoning blocks; benchmark average falls 20.52 points to 48.99); DPO restores format but recovers only part of the loss; GRPO with the language multiplier restores most of it, bringing 30B-A3B to 73.16 on HKCanto-Eval — 1.20 points below its merged checkpoint but now reasoning in Cantonese. The 8B model ends 6.71 points below its merged checkpoint. Transparency is notable: failed RLVR attempts (~775 GPU-hours), a router load-balancing bug, and an admitted DPO-induced formatting regression that makes some reported scores underestimates. Main limitations: evaluation is mostly multiple-choice although the authors themselves state MCQs cannot detect language or generation failures; generation quality rests on eight probes plus qualitative examples; every post-training stage ran once, with no seeds or significance tests; preference data were scored by a proprietary model (Gemini 3.5 Flash). Checkpoints, training environments, and a 13-year Traditional Chinese Common Crawl dataset are released.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Chat-vector merging transfers instruction following at zero training cost but preserves the donor model's reasoning language, unaltered by continuous pre-training (Section 5.1).
- SFT with scarce Cantonese reasoning data substantially damages reasoning: 64.5% empty reasoning blocks at 8B and a 20.52-point HKCanto-Eval drop.
- DPO restores the reasoning-block format (especially at 8B) but recovers only part of the SFT regression (10.19 of 20.52 points).
- RLVR with a multiplicative Cantonese-language and Traditional-Chinese-script reward restores lost performance: the 30B-A3B reaches 73.16 on HKCanto-Eval, 1.20 points below its merged checkpoint, while reasoning in Cantonese.
- In low-resource CPT, final training loss anti-correlates with downstream benchmark score and is not a valid model-selection signal (Table 4).
Strengths
Diagnosis-driven pipeline with quantified failures
Each stage is justified by a measured defect (Table 2): 64.5% empty reasoning blocks at 8B SFT (Table 8), 30B reasoning-trace mean collapsing from 1,442 to 158 tokens after SFT (Table 7), and 24.9% of 30B SFT benchmark generations hitting the 16,384-token cap in repetition loops (Section 6.3).
Multiplicative language/script reward is a genuine methodological contribution
Section 8.2 and Table 11 gate the entire reward on cantofilter language classification (rescaled 0.1+0.9m) and simplified-character density, with an explicit argument for why a multiplier beats an additive term: an additive term would contribute 1/8 of the score and be tradable against correctness.
Unusual transparency about failures and cost
Artifact release and compute accounting
Transferable low-resource CPT lesson
Weaknesses
Evaluation instrument mismatch with the central claim
No statistical uncertainty anywhere
DPO stage confounds later comparisons
8B pipeline is net-negative on the reported benchmark
Proprietary-model dependence for training signal
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is the principal technical report for CantoneseLLM v2: 8B and 30B-A3B reasoning models adapted from Qwen3 for Cantonese, argued to be low-resource in written data despite ~85M speakers. Five claimed contributions map onto the five pipeline stages: CPT on 784M tokens installs local knowledge; chat-vector merging adds instruction following at zero training cost; SFT with scarce Cantonese reasoning traces damages reasoning; DPO restores the reasoning-block format; and a multiplicative language-and-script RLVR reward recovers lost capability — the 30B-A3B ends at 73.16 on HKCanto-Eval, 1.20 points below its merged checkpoint, now reasoning in Cantonese. The claims are documented stage by stage, but the headline capability (Cantonese reasoning quality) is precisely the one the chosen evaluation instrument cannot measure, by the authors' own admission (Section 5.1).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is CantoneseLLM v2?
A family of open-weight Cantonese reasoning language models (8B dense and 30B-A3B mixture-of-experts, based on Qwen3) built by hon9kon9ize through continual pre-training on 784M Cantonese and Hong Kong-related tokens, chat-vector merging, supervised fine-tuning, DPO, and reinforcement learning with verifiable rewards. The checkpoints, training environments, and a 13-year Traditional Chinese Common Crawl dataset are publicly released.
How well does CantoneseLLM v2 perform on benchmarks?
On HKCanto-Eval, the final 30B-A3B model scores 73.16, about 1.20 points below the chat-vector-merged checkpoint that post-training started from, but it can reason and answer in Cantonese, which that checkpoint cannot. The 8B model finishes at 62.80, 6.71 points below its merged checkpoint.
Can the model actually reason in Cantonese?
That is the paper's central claim. After reinforcement learning with a multiplicative Cantonese-language and Traditional-Chinese-script reward, both models produce Cantonese chain-of-thought traces (shown in figures), whereas all earlier checkpoints reasoned in Simplified or Written Chinese even when prompted in Cantonese.
Where can I download CantoneseLLM v2 and its training data?
Model checkpoints are on Hugging Face under the hon9kon9ize CantoneseLLM-v2.0 collection, the RLVR training environments are on GitHub (hon9kon9ize/cantonese-nemo-gym-environments), and the 13-year Traditional Chinese Common Crawl dataset plus translated training blends are on Hugging Face (e.g., jed351).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with '[...sampling gap...]' markers skipping parts of Sections 6.1-6.2, 8.4-9, and two appendix stretches; conclusions about those specific passages rely on cross-references visible elsewhere in the excerpt.
- Only extracted text was available, so figure-based claims (e.g., the Cantonese traces in Figures 7 and 3) are assessed from the surrounding captions and prose, not from the rendered figures.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.