An AI-Powered Culturally Aware Chatbot for Stress Detection and Wellness Support among Pakistani University Students Using NLP and Machine Learning
Muhammad Fahad Bashir, Muhammad Afzal
cs.AI
Quick verdict
Claims a culturally aware ML+LLM stress chatbot for Pakistani students, but the cultural evidence rests on non-Pakistani data and citations are mismatched.
Summary
The paper presents Sukoon, a web system that classifies student stress into three levels and hands the result to an LLM chatbot. A Random Forest (100 trees) is trained on a public survey dataset of 1,100 responses over 20 features (psychological, physiological, environmental, academic, social), using a stratified 70/15/15 split with StandardScaler fitted on training data only. Reported test accuracy is 89.09% with macro F1 0.89; an SVM baseline reaches 88.48%, and the low-stress class shows the lowest recall (0.80). Feature importance ranks blood pressure first (0.156) and teacher-student relationship second (0.100), which the authors frame as a culturally significant finding. The chatbot uses GLM-4.5-Air via OpenRouter with a culturally tailored system prompt supporting English, Urdu, and Roman Urdu, structured by a stepped-care design. Evaluation of the chatbot is limited to simulated inputs and informal testing. Key limitations: the dataset is not Pakistani, so cultural claims remain hypotheses; there is no formal user study; Urdu support is prompt-based rather than a bilingual pipeline; performance rests on a single split without confidence intervals; and the in-text citations are severely mismatched with the reference list.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A Random Forest trained on a public 1,100-response, 20-feature survey dataset classifies students into three stress levels with 89.09% test accuracy and 0.89 macro F1.
- Sukoon, an ML+LLM hybrid chatbot using GLM-4.5-Air via OpenRouter with a culturally tailored system prompt, provides stepped-care wellness support in English, Urdu, and Roman Urdu.
- Feature importance analysis ranks teacher-student relationship second (0.100), framed as evidence for region-aware mental health tooling even though the data is not from Pakistani students.
- The authors claim Sukoon is the first hybrid ML-classification plus culturally adapted LLM dialogue system for Pakistani university students.
Strengths
Leakage-aware, clearly documented ML protocol
Stratified 70/15/15 split with StandardScaler fitted on training data only (§3.2), explicit random_state (§4), per-class F1 and a confusion matrix on the 165-sample test set reported (§5.1, Fig 7)
Includes a same-split baseline comparison
SVM trained on identical splits reaches 88.48% vs RF 89.09% (Table 1), and results are positioned against Zahra et al.'s 87.93% (Fig 6)
Safety-conscious system design
Unusually candid limitations section
Low-resource deployment rationale
Weaknesses
Central cultural claims unsupported by cultural data
Severely corrupted citation apparatus
Chatbot — the actual product — evaluated anecdotally
Statistical reporting stops at point estimates
'NLP' in the title overstates the contribution
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Sukoon is a Flask web system combining a Random Forest stress classifier with an LLM chatbot for Pakistani university students. The paper claims: (1) 89.09% test accuracy and 0.89 macro F1 for three-level stress classification on a public 1,100-response, 20-feature survey dataset (§5.1); (2) culturally adapted wellness dialogue via GLM-4.5-Air through OpenRouter, in English, Urdu, and Roman Urdu, organized by a three-tier stepped-care mapping (§3.4-3.5); (3) that teacher-student relationship ranking second in feature importance (0.100, Table 2) is a culturally significant finding (§5.4); and (4) that this is 'the first hybrid system for this under-served population' (§1, §6). The core tension: cultural framing saturates title and abstract, while §3.1 and §7.2 concede the training data is not Pakistani, and §5.4 itself downgrades the cultural finding to 'a hypothesis.'
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy does the Sukoon stress classifier achieve?
The Random Forest reports 89.09% test accuracy and a 0.89 macro F1 across three stress levels (low, moderate, high) on a 165-sample held-out test set; an SVM comparator reaches 88.48% accuracy with the same macro F1.
Is the training data actually from Pakistani students?
No. The model was trained on a public survey dataset of 1,100 responses. The authors acknowledge in the limitations section that it is not representative of Pakistani students, and plan local data collection with a validated DASS-21 instrument as future work.
Which LLM powers the chatbot and what languages does it support?
GLM-4.5-Air, accessed via the OpenRouter API. It is prompted to converse in English, Urdu, and Roman Urdu using a culturally adapted system prompt; Urdu support is prompt-based rather than a dedicated bilingual NLP pipeline.
Was the chatbot evaluated with real users?
No. Evaluation used simulated inputs covering exam, family, financial, and teacher-related stress, plus informal usability testing. A formal user study with Pakistani students is explicitly listed as future work.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure-embedded values (Fig 1 class distribution, Fig 7 confusion matrix cell counts, Fig 8 full 20-feature importance chart) are not recoverable from text extraction; judgments rely on the prose and tables only.
- The web-app screenshots (Fig 5) and architecture diagrams (Figs 2-4) could not be inspected, so UI-level claims (bilingual labels, five-step form) rest on captions alone.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.