Bangla Sentence Function Classification: Corpus Development, Model Benchmarking, and Interpretability
Swapnil Kundu Argha, Abdullah Al Shafi, Rowzatul Zannat, Shoumik Barman Polok, Abdul Muntakim, Jannatul Ferdousi, M. A. Moyeen
cs.CL · cs.AI · cs.LG
Quick verdict
Introduces a 10k-sentence Bangla function corpus (κ=0.82) and reaches 0.95 F1 with ML ensembles, but reports no transformer baselines or significance tests.
Summary
This paper introduces a corpus of 10,000 Bangla sentences manually annotated for four sentence functions — declarative, interrogative, imperative, and exclamatory — by three annotators using majority voting and consensus resolution, yielding near-balanced classes (~2,500 each) and Fleiss' Kappa of 0.82. The corpus derives from roughly 24,000 scraped sentences (newspapers, blogs, YouTube comments, digitized books) and is released publicly on GitHub. The authors benchmark Bag-of-Words, TF-IDF, and Word2Vec (CBOW/Skip-gram) representations with eight classical classifiers plus two heterogeneous voting ensembles (SLE, DLE), using grid search and four cross-validation schemes; punctuation is deliberately removed so models cannot exploit '?' or '!'. The best configuration, DLE with TF-IDF, achieves 0.95 accuracy and macro-F1 on a 2,000-sentence held-out test set; TF-IDF consistently beats in-corpus Word2Vec, attributed to the small training corpus and discriminative function words. Cross-validation confirms stability (0.92–0.95, std up to 0.024). LIME analysis and four genuine error cases illustrate confusions among rhetorical questions, obligation statements, and exclamatory syntax without markers. Main limitations: no transformer or pretrained-model baselines despite the literature review citing 96–98% results for related Bangla sentence classification; near-identical precision/recall/F1 across Table II suggests heavy rounding, with no per-class metrics or confusion matrices; the 0.94→0.95 ensemble gain lacks significance testing; hyperparameter grids and the 'confidence filtering' step are unreported.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 3
● low 0
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A 10,000-sentence Bangla corpus labeled into four balanced sentence-function classes reaches 'almost perfect' annotation reliability (Fleiss' Kappa 0.82).
- TF-IDF consistently outperforms Word2Vec for sentence-function classification because sparse lexical weighting preserves discriminative function words, especially when embeddings must be trained on the small corpus.
- The double-level heterogeneous ensemble (DLE) with TF-IDF is the best configuration at 0.95 accuracy and macro-F1 on the 2,000-sentence test set, and four cross-validation schemes confirm robustness.
- LIME explanations and error analysis attribute residual errors to genuine linguistic overlap: rhetorical questions, obligation modality, and exclamatory syntax without exclamation marks.
Strengths
New public resource for an under-resourced language
10,000 Bangla sentences with four-way function labels, near-balanced (2,470–2,532 per class, Table I), Fleiss' Kappa 0.82, built through a documented majority-vote-plus-consensus pipeline (Algorithm 1) and released on GitHub.
Leak-resistant, fairness-conscious preprocessing
Punctuation is removed so models cannot shortcut the task via '?'/'!' (Sec. III-B), wh-words are deliberately retained, and one shared tokenizer is used across BoW/TF-IDF/Word2Vec for fair comparison.
Broad, honestly cross-validated classical benchmark
Concrete linguistic error analysis
Weaknesses
No neural or transformer baselines
Uninformative metric reporting
Untested headline margins
Corpus construction opacity
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper makes a resource-plus-benchmark contribution for Bangla NLP: (1) a corpus of 10,000 Bangla sentences annotated by three annotators into four sentence functions (declarative, interrogative, imperative, exclamatory), nearly balanced at ~2,500 per class, with Fleiss' Kappa 0.82 (Sec. III-A, Table I); (2) a benchmark of BoW, TF-IDF, and Word2Vec (CBOW/Skip-gram) representations across eight classical classifiers plus two heterogeneous voting ensembles, SLE and DLE (Secs. III-C/III-D); and (3) interpretability via LIME plus a qualitative error analysis (Sec. IV, Table IV, Fig. 5). The headline claim is that DLE with TF-IDF reaches 0.95 accuracy and macro-F1 on a 2,000-sentence held-out test set, with TF-IDF consistently beating Word2Vec. The corpus is released on GitHub.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the Bangla Sentence Function Classification corpus?
A public dataset of 10,000 Bangla sentences labeled into four sentence functions (declarative, interrogative, imperative, exclamatory) by three annotators using majority voting and consensus discussion, with inter-annotator Fleiss' Kappa of 0.82 and roughly 2,500 sentences per class; it is released on GitHub (AbdullahRatulk/Bangla_Sentence_Function_Classification_Corpus).
What accuracy does this paper achieve?
The best model, a double-level voting ensemble (DLE) using TF-IDF features, reaches 0.95 accuracy and 0.95 macro-F1 on a held-out test set of 2,000 sentences; strong single models such as linear SVM, random forest and gradient boosting reach about 0.94 with TF-IDF.
Does the paper use deep learning or models like BERT?
No. The benchmark covers classical machine learning (Naive Bayes, linear SVM, logistic regression, decision tree, Ridge, AdaBoost, gradient boosting, random forest) plus voting ensembles over BoW/TF-IDF/Word2Vec features. Transformer models are explicitly listed as future work, which is the review's main criticism.
Is the code and data available?
The corpus itself is publicly released on GitHub. The paper does not state that training/evaluation code or exact hyperparameter settings are published, so exact reproduction of the result tables may require reimplementation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Bangla conjuncts and vowel signs in the extracted example sentences are partially garbled by PDF extraction; the English glosses supplied by the paper were relied on for linguistic examples.
- Figure interiors (Figs. 1-5) are not rendered in the extraction, only captions and scattered fragments; assessments of figures rely on the body text describing them.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.