Enemray: Toward Capable Language Models for Hassaniya
cs.CL · cs.AI
Quick verdict
Gemma 4 E4B adapted to Hassaniya via CPT+graft+SFT tops En→Ha MT, but auto-metric-only eval, Gemini-derived data, and withheld weights limit confidence.
Summary
The paper introduces Enemray, a Hassaniya-language assistant built by adapting Gemma 4 E4B (~8B parameters) through a three-stage pipeline that separates language acquisition from behavioral specialization. Stage I runs LayRA-style continual pretraining on 11,629 documents (20.9M tokens) of web-collected Hassaniya, updating LoRA adapters in 12 of 42 decoder layers. Stage II grafts the scaled language update (γ=0.7) into the instruction-tuned checkpoint via task arithmetic. Stage III fine-tunes all layers on 46,532 conversations, mixing newly constructed Hassaniya data (context-guided translation pairs and dialogues generated with Gemini, human-verified, plus correction data and poetry), selected public datasets, and a 273-conversation general-capability replay set (~17% of target tokens). On the Alexandria (Mauritania) test set, Enemray scores 12.64 spBLEU / 29.42 chrF++ for English→Hassaniya, best among compared open and proprietary models, and the highest Hassaniya→English spBLEU (29.52), though Gemini 3.5 Flash Lite leads chrF++ (49.93 vs 48.99). On AlexandriaX Subtask 3 error detection it attains the top overall score (17.04) but the lowest exact-match F1 due to over-broad spans (46.9% coverage vs 19.5% gold). General capabilities largely persist: GSM8K 89.4, MMLU-Pro 62.8, BFCL 91.2; HumanEval drops to 74.4 but recovers to 87.8 with the reasoning mode. Limitations: fully automatic evaluation without native-speaker assessment, no ablations (e.g., of γ=0.7), single-run results without variance estimates, and the checkpoint is withheld for licensing reasons.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 4
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Layer-selective continual pretraining plus task-arithmetic grafting into the instruction-tuned checkpoint yields the best English→Hassaniya translation among compared open and proprietary models (12.64 spBLEU vs 8.83 for Gemma 4 E4B-it, Table 7).
- Enemray attains the highest overall score on Mauritanian translation error detection (AlexandriaX Subtask 3: 17.04), while retaining most general capabilities of its base model.
- General capabilities are largely preserved: 95.5% of base GSM8K accuracy, 91.3% of MMLU-Pro, near-unchanged BFCL function calling, and HumanEval recovery from 74.4 to 87.8 with the reasoning mode enabled.
Strengths
Unusually complete implementation documentation
Table 5 lists complete hyperparameters for both stages (LoRA rank 16/α 32, lr 5e-5, batch 16, seeds); §4.2 verifies the FP32 merge reproduces adapted-model logits to within 1.7e-4 — a sanity check rarely reported in grafting work.
Honest negative reporting
Table 8 shows Enemray with the lowest exact-match F1 (2.79) and 46.9% span over-marking; §5.3 openly diagnoses the HumanEval drop (88.4→74.4) and its partial recovery with reasoning (87.8, Figure 6).
Valuable resource landscape and licensing transparency
Data provenance and contamination hygiene
Clean evaluation protocol
Weaknesses
No human or native-speaker evaluation despite the core claim being language quality
Gemini circularity in training data vs. benchmarking
No ablations for the pipeline's core choices
Statistically thin result reporting
Irreproducible outside the authors
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Enemray (arXiv:2609.14829) is a single-author system report describing a Hassaniya-centric assistant built on Gemma 4 E4B (~8B total / 4.5B effective parameters). Hassaniya is the Arabic variety of Mauritania and the western Sahel — primarily oral, orthographically variable, and sparsely represented in NLP. The paper claims three headline results: (1) best English→Hassaniya translation among all compared open and proprietary models on the Alexandria (Mauritania) test set — 12.64 spBLEU / 29.42 chrF++ vs 8.83 / 26.09 for the base model (Table 7); (2) highest overall score on Mauritanian MT error detection, AlexandriaX Subtask 3 (17.04, Table 8); (3) retention of most general capabilities of the instruction-tuned base (GSM8K 89.4 vs 93.6; MMLU-Pro 62.8 vs 68.8; BFCL 91.2 vs 92.0; HumanEval 74.4 vs 88.4, Figure 5). The algorithms themselves are prior work — LayRA-style layer-selective CPT [14], task-arithmetic grafting [17, 21], replay-regularized SFT [16] — so the novelty rests on their composition and on the data constructed for an extremely low-resource language.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Enemray?
Enemray is a language model fine-tuned from Gemma 4 E4B to converse, translate, and answer in Hassaniya, the Arabic variety spoken mainly in Mauritania. It is built in three stages: layer-selective continual pretraining on ~20.9M tokens of web-collected Hassaniya, grafting the learned language update into the instruction-tuned model (γ=0.7), and supervised fine-tuning on 46,532 conversations with general-capability replay.
Can I download or use Enemray?
No. The paper states the checkpoint will not be publicly released because several training datasets (e.g., Alexandria under CC BY-NC, PALM under non-commercial terms) restrict derivative redistribution. The authors say they are building a license-clean version with locally sourced data for future open release. No code release is mentioned either.
How good is Enemray at Hassaniya translation?
On the Alexandria (Mauritania) test set (1,000 pairs per direction), it scores 12.64 spBLEU / 29.42 chrF++ for English→Hassaniya — the best among compared models including GPT-5.6 Luna, Gemini 3.5 Flash Lite, and Qwen 3.8 Flash. In the Hassaniya→English direction it has the highest spBLEU (29.52) but slightly trails Gemini 3.5 Flash Lite on chrF++ (48.99 vs 49.93).
What is Hassaniya?
Hassaniya (Ḥassāniyya) is a Western Arabic variety spoken across Mauritania and the western Sahara/Sahel, where it functions as a lingua franca. It is primarily oral, shows heavy contact with Classical/Modern Standard Arabic, Berber languages, and French, and is sparsely represented in NLP datasets.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Arabic-script Hassaniya examples (Figure 1, Table 9) are garbled by PDF text extraction; output orthography and generation quality could not be directly inspected.
- Figures appear only as caption text with embedded numbers; all numeric claims were verified against tables and captions, not rendered plots.
- Author affiliation, funding, and acknowledgments are absent from the extracted text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.