Towards Stress-Aware Sentence-Level Filipino G2P With Weakly-Supervised ByT5 Fine-Tuning
Lorenz Bernard Marqueses, Paulo Grane Gabriel Silva, Chastine Cabatay, Ericson Adler Tan, Ann Franchesca Laguna
cs.CL
Quick verdict
Claims ByT5 fine-tuning yields 0.54% PER stress-aware sentence-level Filipino G2P with 83-89% stress accuracy; convincing results, but evaluation is narrow.
Summary
The paper addresses stress-aware grapheme-to-phoneme (G2P) conversion for Filipino at sentence level, where lexical stress is not written yet changes word meaning and must be inferred from context. The authors fine-tune a ByT5 model initialized from CharsiuG2P weights on word-level WikiPron data plus three LLM-annotated sentence-level datasets: Tatoeba (5,662 training sentences), NewsPH-NLI (17,742), and two synthetic sets — a naive one (21,993) and a Wiktionary-guided one (47,402) built from 1,511 semantically ambiguous words. Gemini models select the context-appropriate pronunciation for ambiguous words; rule-based G2P handles the rest. On a 1,173-sample manually-corrected test set, the best configuration reaches 0.54% pooled PER and 2.50% CER, versus 19.74% PER for zero-shot CharsiuG2P and 11.70% for Epitran; pairwise Wilcoxon tests with Holm-Bonferroni correction find all dataset combinations significantly different except configs (5) vs (6) on PER/PFER. Stress classification on ambiguous words yields 83-89% per class, weakest for malumi (F1 0.76); about 30% of nonstandard words are misclassified as malumay. Main limitations: training labels are unvalidated LLM output; the gold test set draws only from Tatoeba and synthetic data, excluding the longer NewsPH sentences; and no fine-tuned word-level baseline isolates the contribution of sentence context.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Fine-tuning a CharsiuG2P-initialized ByT5 on LLM-annotated sentence-level Filipino data reduces pooled PER from 19.74% (zero-shot base) to 0.54% on a 1,173-sample manually-corrected test set, with CER of 2.50%.
- The Wiktionary-guided synthetic dataset's measurable benefit is stress-marker placement (CER 2.83% → 2.50%, p=0.0015), not phonetic accuracy (PER/PFER unchanged significantly).
- The best model correctly classifies 83-89% of the four main Filipino stress classes on semantically ambiguous words, with malumi weakest (F1 0.76) and ~30% of nonstandard words misclassified as malumay.
- ByT5-based sentence-level Filipino G2P offers strong potential for homograph disambiguation, though misplaced stress on homographs remains the dominant error source.
Strengths
Clear task motivation for a genuinely under-explored problem
§1 motivates sentence-level context via lexical stress minimal pairs (bukas 'tomorrow' vs 'open') and affix-driven stress shifts (sulat → nagsulat); §2.2 confirms stress-aware sentence-level Filipino G2P is 'mostly unexplored territory'.
Careful, layered data construction with a built-in ablation of annotation guidance
§3.1 filters 4,231 ambiguous WikiPron words down to 1,511 semantically ambiguous ones (multiple stress classes + distinct Wiktionary senses); §3.2 contrasts a naive synthetic set (LLM guesses pronunciations) with a definition-guided set, enabling a direct test of whether guidance helps.
Statistically disciplined model comparison
Gold-standard evaluation plus a prosody-specific metric
Honest error analysis and released artifacts
Weaknesses
Gold test set covers only two of four data sources
Weak-supervision noise is never quantified
Missing fine-tuned word-level baseline
Best-model framing is slightly overstated
Stress evaluation restricted to ambiguous words, with thin minority-class support
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets stress-aware, sentence-level grapheme-to-phoneme (G2P) conversion for Filipino. Although Filipino orthography is shallow, stress diacritics are dropped in normal writing while stress is lexically distinctive (bukas /ˈbukas/ 'tomorrow' vs /buˈkas/ 'open'), and affixation shifts stress unpredictably (sulat → nagsulat), so correct transcription requires sentence context (§1). Three central claims are made: (1) fine-tuning a CharsiuG2P-initialized ByT5 on LLM-annotated sentence data cuts pooled PER from 19.74% to 0.54% on a manually-corrected test set (Table 3); (2) a Wiktionary-guided synthetic set improves stress-marker placement, visible in CER; (3) the model classifies 83-89% of the four main Filipino stress classes on ambiguous words, supporting homograph disambiguation (§5.3).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy does this Filipino G2P model achieve?
On a manually-corrected test set of 1,173 sentences, the best model reaches about 0.54% pooled phoneme error rate (PER) and 2.50% character error rate (CER). For comparison, the zero-shot multilingual CharsiuG2P base model scores 19.74% PER and the rule-based Epitran system 11.70% PER on the same set.
Why is Filipino G2P hard if the writing system is shallow?
Filipino spelling maps to sounds fairly directly, but stress is not written down while being lexically distinctive: bukas with penultimate stress means 'tomorrow', with final stress means 'open'. Picking the right pronunciation requires sentence context, and Filipino's many affixes shift stress in ways that are hard to capture with rules.
How was the training data for this model created?
The authors scraped word pronunciations from Wiktionary via WikiPron, then used Gemini LLMs to annotate sentence corpora (Tatoeba, NewsPH-NLI) and to generate synthetic sentences: for words with multiple possible pronunciations, the LLM chooses the one fitting the sentence context, while a rule-based G2P handles unambiguous words. A second synthetic set was generated from 1,511 words whose pronunciations correspond to different meanings.
Can this model disambiguate Filipino homographs?
Partially. Evaluated only on semantically ambiguous words, the model picks the correct stress class 83-89% of the time across the four main Filipino stress patterns, performing worst on the malumi class (F1 0.76). Misplaced stress on homographs is still the dominant error type, so the authors frame homograph disambiguation as a promising application rather than a solved problem.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- IPA glyphs appear in the extraction as ASCII-like substitutes (e.g., P for the glottal stop, N for the velar nasal), consistent with the paper's own Table 1 rendering; exact Unicode phonetic symbol fidelity could not be verified.
- Figures 1, 2 and 4 are present only via captions and surrounding prose in the extraction; the visual content of the pipeline diagram, architecture overview, and UMAP plot could not be inspected.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.