YNU-HPCC at SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Using Multiple Prediction Headers
Hao Yang, Jin Wang, Xuejie Zhang
cs.CL · cs.AI
Quick verdict
SemEval-2025 Task 11 paper: translate-to-English+encoder scores 0.44; workable system but reporting is self-contradictory (RoBERTa vs DeBERTa, swapped labels).
Summary
This SemEval-2025 Task 11 (subtask A) system paper describes YNU-HPCC's entry in multi-label emotion detection. The pipeline machine-translates all input into English with Google Translate, then fine-tunes a Transformer encoder (RoBERTa in the abstract, DeBERTa in the methodology — the paper contradicts itself) with modified prediction heads, Focal Loss, and R-Drop regularization. The official score is 0.44. The paper reports per-language and per-emotion F1 for 27 languages (Table 5), ranging from 0.82 (Hindi) to 0.05 (Emakhuwa), and an ablation comparing single-emotion versus multi-emotion prediction heads (Table 4). Two headline findings are claimed: single heads beat six simultaneous heads, and training on uniformly translated English data beats original data. The paper's main limitation is its own reporting: Table 4's caption admits its variant labels were swapped, and as printed the table appears to contradict claim (I); claim (II) has no supporting experiment anywhere in the text; the dataset statistics in Section 3.3 (joy 10,165, sadness 7,305) contradict Table 3 (13,182 / 12,311); Section 4 mentions comparing 'two emotions' predictions; and 29 languages are claimed versus 27 listed. No hyperparameters, seeds, significance tests, or baseline comparisons are provided. Code is released on GitHub.
Review complete
Assessment at a glance
6
risk flags identified
● high 3
● medium 3
● low 0
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A translate-to-English pipeline with a fine-tuned Transformer encoder, modified prediction heads, Focal Loss, and R-Drop achieved an official SemEval-2025 Task 11 subtask A score of 0.44 across all submitted languages.
- A single prediction head per emotion outperforms six heads predicting all six emotions simultaneously (claim I).
- Training on uniformly translated English data yields better results than training on the original multilingual data (claim II).
- Focal Loss plus R-Drop stabilizes training and improves performance on underrepresented emotions, per the Table 4 ablation.
Strengths
Broad low-resource coverage with honest failure reporting
Table 5 reports per-language and per-emotion F1 for 27 languages including Emakhuwa, Tigrinya, Oromo, and Somali, and does not hide catastrophes (Emakhuwa macro-F1 0.0457, Tigrinya 0.2822, Somali disgust 0.0662) alongside strong results (Hindi 0.8221, Marathi 0.8199).
Practical translate-then-classify pipeline for dialect-heavy languages
Section 1 motivates the design concretely: mBERT handles 100+ languages but performs poorly on Algerian/Moroccan Arabic and Nigerian Pidgin, so Google Translate preprocessing plus English-only encoders is a defensible engineering choice (Sections 1, 3).
Ablation grid over head design and training objectives
Code released
Explicit accounting of label imbalance
Weaknesses
The base model is unidentifiable
Table 4 is self-undermining and the discussion is circular
Headline claim (II) is never tested
Numerical inconsistencies throughout
No statistical rigor or training detail
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a shared-task system paper for SemEval-2025 Task 11, subtask A (multi-label detection of anger, disgust, fear, joy, sadness, surprise), from the YNU-HPCC team at Yunnan University. The system machine-translates all input into English with Google Translate and fine-tunes a Transformer encoder with modified prediction heads, Focal Loss, and R-Drop. The official score is 0.44. The abstract advances two analysis claims: (I) a single prediction head outperforms six heads predicting six emotions simultaneously, and (II) training on uniformly translated English data outperforms the original dataset. The scope is narrow — a system description — but the claims are phrased as general findings, so they are evaluated as such here.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What score did YNU-HPCC achieve in SemEval-2025 Task 11 subtask A?
The paper reports an official score of 0.44, evaluated across all submitted languages. Per-language macro F1 in Table 5 ranges from 0.82 (Hindi, the best) down to 0.05 (Emakhuwa, the worst), with 27 languages listed.
How does the system handle low-resource languages and dialects?
Instead of training a multilingual model, the entire dataset is machine-translated into English with Google Translate, then English-only Transformer classifiers fine-tuned with Focal Loss and R-Drop predict the six emotions. The abstract claims this beats training on the original data, but no such comparison experiment appears in the paper.
Is the code for this paper available?
Yes — the abstract links to github.com/BGWH123/Semeval-2025-task11. However, the paper itself lacks hyperparameters and contains internal contradictions (RoBERTa vs DeBERTa, swapped ablation labels), so the repository is the only reliable reference for what was actually run.
Did single prediction heads really beat multi-head prediction in this paper?
The abstract claims so, but Table 4's caption admits its variant labels were swapped, and the printed F1 numbers show the opposite column group winning for all six emotions (e.g., anger 0.701 vs 0.325). The paper never resolves this contradiction, so the claim should be treated as unsupported.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 is garbled in the extraction; the architecture assessment relies on the prose of Sections 3.2 and 5.
- Table 4's cell alignment may be imperfect in the extracted text, though the label-swap admission in its caption is verbatim; Table 5 contains a literal 'Nan' cell (Afrikaans).
- Test-set sizes, decoding thresholds, and the official score's aggregation are absent from the text, so 0.44 could not be reconciled with the listed per-language macro-F1s (mean ≈ 0.497).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.