Improving Language Identification for Code-Switched Utterances with Integer Linear Programming
Joanna Radoła, Josep Maria Crego, François Yvon
cs.CL
Quick verdict
Reformulates MaskLID code-switch LID as an ILP with a word-aware backbone; large EM gains, well ablated, but only 125 Latin-script languages.
Summary
The paper revisits MaskLID, a training-free post-processing method that turns any sentence-level language identifier (LID) into a code-switched (CS) LID, and improves it on two fronts. First, the authors show that FastText-based backbones such as GlotLID produce unreliable word-level posteriors (mean gold-label probability 0.31 at word level vs 0.98 at sentence level, Figure 1), and they train LiteLID-v2, a 125-label Latin-script FastText model augmented with single-word training instances, which sharply improves short-segment and word-level behavior. Second, they reformulate MaskLID's greedy iterative masking as an Integer Linear Program (ILP4LID) that explicitly maximizes word-language association scores under interpretable constraints (one language per word, at most K languages, minimum span length, top-M masking, rank-weighted objective with per-language penalty). On a test suite of eight CS pairs spanning ten languages plus balanced monolingual data, the two changes raise overall Exact Match from 0.54 (GlotLID+MaskLID) to 0.71 (LiteLID-v2+ILP4LID AVG), with pair-level gains up to +0.44 EM for Turkish-English; McNemar's tests confirm significance for the AVG and MONO configurations, while the CS-only configuration is borderline (p=0.0498) and fails on monolingual data (EM 0.04). Limitations include the restricted 125-language label set, a 10-20x runtime increase (commercial Gurobi, though HiGHS is supported), config-dependent behavior, and Latin-script-only evaluation. Code and data are released.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- MaskLID's central weakness is its reliance on word-level scores from sentence-trained LIDs, which are unreliable (mean gold-label probability 0.31 at word level under GlotLID vs 0.98 at sentence level).
- Training a backbone on full sentences plus single words (LiteLID-v2) largely repairs MaskLID's code-switched performance without changing the masking algorithm.
- Reformulating the word-language assignment as an ILP yields better global solutions than greedy masking, with interpretable and extensible constraints.
- Together the changes raise test overall Exact Match from 0.54 to 0.71 (AVG configuration), with CS pair gains up to +0.44 EM for Turkish-English, significant by McNemar for the AVG and MONO configurations.
Strengths
Convincing failure-mode diagnosis
Section 3.3 and Figure 1 quantify the core problem: GlotLID assigns the gold label a mean probability of 0.98 at sentence level but only 0.31 at word level; Appendix B extends this to top-M lists (gold label in top-10 rises from 78% to 94% for English and 65% to 96% for Hindi with the new model).
Well-controlled backbone comparison
LiteLID-v0 (Section 3.4, Figure 2) reproduces GlotLID's training recipe on 125 labels and is nearly indistinguishable from GlotLID, isolating word-level data augmentation — not the label-set reduction — as the active ingredient; LiteLID-v2 gains 0.14 absolute F1 on 6-10 char segments.
Principled formulation with genuine ablation
Honest significance testing and error analysis
Reproducibility posture
Weaknesses
Tiny dev set for heavy configuration search
No external CS-LID baseline on the new benchmark
Coverage regression vs the motivating use case
Config-dependent deployment
Runtime and benchmark curation gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets code-switched (CS) language identification: given an utterance mixing two languages, return the exact set of languages present. It revisits MaskLID (Kargaran et al., 2024b), a training-free iterative-masking wrapper around sentence-level LIDs, and claims three contributions: (a) diagnosing MaskLID's overreliance on unreliable word-level scores and fixing it with a retrained backbone (LiteLID-v2); (b) reformulating the assignment as an Integer Linear Program (ILP4LID) with interpretable constraints; (c) large gains across eight CS pairs over ten languages. Claims are explicitly scoped to Latin-script languages (125 labels) and two-language mixtures, and the Limitations section is candid about both boundaries.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ILP4LID and how does it differ from MaskLID?
ILP4LID is a reformulation of MaskLID's code-switched language identification as an integer linear program: instead of greedily masking words round by round, it assigns every word to at most one language while maximizing total word-language scores under interpretable constraints (at most K languages, minimum span length, top-M masking). MaskLID's greedy passes leave fragments too short to classify, so on the paper's test data 23% of words stay unassigned, versus 97% assigned by ILP4LID.
How much better is ILP4LID than MaskLID?
On the paper's benchmark of eight code-switched language pairs over ten languages, overall Exact Match rises from 0.54 (GlotLID backbone plus MaskLID) to 0.71 with the full method (LiteLID-v2 plus ILP4LID, AVG configuration). Per-pair code-switched EM gains reach +0.44 for Turkish-English and +0.35 for Nepali-English; McNemar tests confirm significance for the AVG and MONO configurations.
What are the main limitations of this approach?
The released backbone covers only 125 Latin-script languages (GlotLID covers 2000+), inference is roughly 10-20 times slower than MaskLID (13-25 seconds per 80 sentences on a desktop CPU), exact results were obtained with the commercial Gurobi solver (open-source HiGHS is supported but unreported), and the CS-specialized configuration fails on monolingual text (EM 0.04), so the AVG configuration is recommended in practice.
Do I need a GPU to run ILP4LID?
No. The backbone is a FastText model trainable in about 8 hours on a single CPU, and the ILP is solved on CPU; the authors suggest distributing sentences across cores. The main cost is solver time, roughly 10-20x MaskLID's runtime.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Text inside Figures 1-4 is garbled by PDF extraction (repeated axis strings); figure-based statements rely on captions and in-text reported means (GlotLID sentence 0.98 / word 0.31; LiteLID-v2 0.96 / 0.50).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.