Reinforcement Learning for improving Large Language Models' Catalan text simplification capabilities
Arnau Ayguadé Domingo, Stefan Bott, Horacio Saggion
cs.CL · cs.AI
Quick verdict
GRPO with a SARI-plus-penalties reward improves Catalan simplification SARI, but gains are metric-aligned, modest, and human evaluation is missing.
Summary
This paper post-trains IberianLLM-7B-Instruct with GRPO for sentence-level automatic text simplification in Catalan, a low-resource language lacking dedicated simplification models. The novel reward combines normalized SARI with a copy penalty and an exponentially decaying length penalty beyond five excess words (alpha=4), motivated by an observed over-generation behavior that inflated SARI. Three variants were trained on English ASSET and on machine-translated Spanish and Catalan versions (BLEURT 0.7-0.9 filtered). All variants improve the Catalan ASSET test split by 2-2.7 SARI points and suppress over-generation; on the 304-sentence filtered iDEM benchmark, only the English-trained variant improves significantly (+1.19), and translated variants show no significant gains. The authors candidly note single runs, possible data leakage, and SARI's weaknesses. The main caveats: training reward and evaluation metric are both SARI, so gains largely follow by construction; no human evaluation and no supervised fine-tuning control test the claimed RL advantage over SFT; the +1.19 effect is small. Still, methodological transparency, honest statistical uncertainty reporting, and a real accessibility motivation for a low-resource language make this worthwhile, cautious reading.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- GRPO post-training with a novel SARI-plus-penalties reward suppresses over-generation and improves IberianLLM-7B-Instruct's Catalan sentence simplification (Sections 3-4).
- All three training-language variants improve the Catalan ASSET test split by 2-2.7 SARI points (Table 1).
- Only the English-ASSET-trained variant yields a statistically significant iDEM improvement over the original model (+1.19 [0.18, 2.20]; Table 2).
- Post-training on machine-translated ASSET (Catalan or Spanish) fails to improve the out-of-domain iDEM benchmark, with the separation lying between translated and untranslated corpora rather than between languages (Section 5).
Strengths
Reward design attacks a real, observed failure mode
Section 3.1 documents that over-generated outputs scored higher SARI than truncated ones; the length penalty (Eq. 2, tau=5, alpha=4) and copy penalty (Eq. 1) are directly motivated, and training cut length-penalty triggers from ~50% to 0-5% (Section 4).
Transparent, unusually honest reporting
Section 3 admits the reference-based reward forfeits RL's main advantage over SFT; Section 6 flags the ASSET split-leakage risk itself; Appendix B cautions that the EN-vs-Original CI lower bound (0.18) implies a possibly very small true effect.
Proper paired significance testing on the out-of-domain benchmark
Genuine low-resource impact path
Weaknesses
Metric circularity: evaluation is the training objective
No SFT baseline
Abstract slightly overclaims the iDEM result
Restricted data pipeline cuts both ways
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is an empirical, single-model study of RL-based post-training (GRPO) for sentence-level automatic text simplification (ATS) in Catalan. Claims: (1) a novel reward — normalized SARI minus a copy penalty, times an exponential length penalty (Eqs. 1-3, tau=5, alpha=4) — steers GRPO toward a targeted simplification style and suppresses an observed over-generation behavior; (2) post-training on English ASSET transfers to Catalan, with all three variants improving the Catalan ASSET test split by 2-2.7 SARI points (Table 1); (3) translated-ASSET variants (Spanish/Catalan) do not significantly improve the out-of-domain iDEM benchmark (Table 2). The scope is honest: the authors repeatedly qualify results as within a 'limited experimental setting'.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does this paper do?
It improves a 7B language model (IberianLLM) for sentence simplification in Catalan using reinforcement learning (GRPO), with a reward function built from the SARI metric plus penalties against copying the input and generating overly long outputs.
What were the main results?
All model variants improved the Catalan ASSET test SARI by about 2-2.7 points and stopped over-generating text. On the harder 304-sentence Catalan iDEM benchmark, only the model trained on English ASSET improved significantly (+1.19 SARI); models trained on machine-translated ASSET did not improve significantly.
Why did training on English data work better than training on Catalan data for a Catalan task?
The authors' results show the separation lies between the translated and untranslated training corpora rather than between languages; they conjecture that machine translation altered the simplification characteristics of the corpus, but state their experiments do not establish why.
What are the main limitations?
Per the authors: SARI is a weak proxy for human-judged simplification quality and was also the training reward; possible train/test overlap in the ASSET split; BLEURT filtering removed deep-deletion simplifications; each language variant is a single training run, so variance across runs is unknown.
Is code and the model available?
Yes. The paper footnotes code at github.com/arnauad/simplification and the final model at huggingface.co/arnauad/IberianLLM-ASSET-GRPO. The full system prompt and few-shot examples are listed in Appendix A.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- GRPO training hyperparameters (group size, learning rate, batch size, rollouts per prompt) are not stated anywhere in the extracted text; this is an absence in the paper, not truncation of the extraction.
- It is not stated how many (machine-translated) references per sentence were used when computing SARI on the Catalan ASSET test set.
- The extracted text is otherwise complete: title, abstract, Sections 1-6, Acknowledgments, references, and both appendices are present, with only routine PDF line-break artifacts.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.