Contextual Tamil Spelling and Grammar Correction Using Progressively Fine-Tuned Sequence-to-Sequence Transformers
Karthikeyan A, Jaya Nirmala S, Sangeetha Sivanesan, Indhu R, Pranav Kumar, Bharat Jude Johnson, Vishnu Ram
cs.CL
Quick verdict
Fine-tunes mT5/mBART-50 on synthetic Tamil pairs; best 69.3% top-1 with clean staged ablations, but all data synthetic and no code release limits impact.
Summary
The authors fine-tune mT5-small and mBART-50 for end-to-end Tamil spelling and grammar correction, arguing that prior rule-based, n-gram, and retrieval-plus-ranking systems cannot fix contextual errors such as subject–verb agreement, tense consistency, and cross-word sandhi. From 1.2 million cleaned Tamil Wikipedia sentences they build a synthetic corpus of up to 657,720 noisy–clean pairs spanning ten error categories, and train both backbones under an identical four-stage schedule: surface noise (v2), contextual grammar (v3), single-site sandhi (v4), multi-site sandhi (v5). Evaluation uses a 1,000-sentence balanced diagnostic set (200 per category), verified disjoint from training by exact-hash and 5-gram overlap checks, with a copy baseline fixed at 20.0%. Best model mBART-50 v5 reaches 69.3% top-1 exact match (87.5% sandhi, 43.5% subject–verb, 74.0% identity) versus 63.1% for mT5 v5. The ablation shows subject–verb accuracy rising from 1.0% to 52.5% only when contextual pairs are introduced, sandhi from 0% to 83.5% only with multi-site pairs, and identity accuracy falling monotonically as sandhi recall rises. Prompted Tamil-LLaMA-7B manages 19.0% zero-shot and 24.7% three-shot. Limitations: entirely synthetic data with Wikipedia domain bias, a contested gold convention for optional sandhi, an admitted size–architecture confound, and no code or data release.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Fine-tuning mT5-small and mBART-50 under an identical four-stage progressive schedule (surface noise, contextual grammar, single-site sandhi, multi-site sandhi) yields 69.3% top-1 exact-match accuracy for mBART-50 v5 on a 1,000-sentence balanced diagnostic set verified disjoint from training.
- Each error capability emerges only with supervision of the matching kind: subject–verb agreement rises from 1.0% to 52.5% only when contextual pairs are introduced at v3, and cross-word sandhi from 0% to 83.5% only with multi-site pairs at v5 (mT5 trajectory).
- Sandhi recall is paid for monotonically in identity accuracy, a precision–recall trade-off the paper claims prior Tamil spell-correction literature has not reported.
- Prompted Tamil-LLaMA-7B-Instruct does not transfer to this task (19.0% zero-shot, statistically indistinguishable from the 20.0% copy baseline; 24.7% three-shot), indicating task-specific supervision rather than model scale is what the problem currently requires.
Strengths
Evaluation hygiene above the field's norm
The 1,000-sentence diagnostic set (Section 3.7) is verified disjoint via exact-match hash plus 5-gram Jaccard rejection rather than assumed unseen, is category-balanced (200 per slice), and ships with a copy baseline fixed at 20.0% by construction; by contrast, the closest prior work (TamilMayangoliSpell) evaluates on a 10% in-distribution split of its own induced distribution with no no-edit items.
Clean curriculum ablation isolating supervision effects
Both backbones share one four-stage schedule (Section 4.2, Table 3), letting the paper attribute specific gains to data composition rather than architecture: subject–verb agreement 1.0% to 52.5% at v3, sandhi 0% to 63.0% to 83.5% at v4/v5 (mT5 columns of Table 5), with replay pairs used against catastrophic forgetting and v4/v5 both resuming from v3.
Novel, quantified sandhi–identity trade-off
Statistically careful LLM comparison
Weaknesses
Entirely synthetic errors from a single domain
Gold convention contaminates the identity/sandhi story
Confounded architecture comparison and zero artifact release
Coverage gaps and an unresolved forgetting regression
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper formulates Tamil spell and grammar correction as end-to-end sequence-to-sequence generation, positioning itself against rule-based systems, statistical n-gram and MED methods, and hybrid retrieval-plus-re-ranking pipelines that cannot repair contextual errors (subject–verb agreement, tense consistency, cross-word sandhi). It claims: (1) fine-tuning mT5-small and mBART-50 on up to 657,720 synthetic noisy–clean pairs under an identical four-stage schedule reaches 69.3% top-1 exact match for mBART-50 v5; (2) capabilities emerge only with matching supervision — subject–verb agreement 1.0% to 52.5% at v3, sandhi 0% to 63.0% to 83.5% at v4/v5 (mT5 columns of Table 5); (3) sandhi recall is paid for monotonically in identity accuracy (87.0% to 72.5% across mT5 stages); (4) prompted Tamil-LLaMA-7B-Instruct does not transfer (19.0% zero-shot vs a 20.0% copy baseline; 24.7% three-shot). The claims are mostly supported within the paper's self-declared synthetic-only scope, with two material caveats discussed below.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy does this Tamil spelling and grammar correction model achieve?
The best model, mBART-50 v5, reaches 69.3% top-1 exact-match accuracy on a 1,000-sentence balanced diagnostic set: 87.5% on cross-word sandhi, 43.5% on subject–verb agreement, 74.0% on leaving already-correct sentences unchanged. A trivial system that copies its input scores 20.0% by construction.
Is the training data real or synthetic?
Entirely synthetic. Noise is injected programmatically into 1.2 million clean Tamil Wikipedia sentences, producing up to 657,720 noisy–clean pairs across ten error categories, because no publicly available corpus of authentic annotated Tamil spelling errors exists.
Can Tamil-LLaMA do Tamil spell correction out of the box?
No. Prompted Tamil-LLaMA-7B-Instruct scores 19.0% zero-shot — statistically indistinguishable from the 20.0% copy baseline — and 24.7% with three examples, versus 69.3% for the fine-tuned mBART-50 model. Its main failure mode is rewriting parts of the sentence that needed no correction.
Is code or data available for this paper?
No. The paper states that its pipeline, data, and models are not yet publicly released, explicitly contrasting itself with the related TamilMayangoliSpell work, whose pipeline, data, and models are public.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Tamil glyph sequences in the extracted text show vowel-sign reordering typical of PDF extraction (visible in the worked examples of Section 1); all numbers, tables, and English prose extracted cleanly, and no judgment in this review depends on the garbled strings.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.