Orthogonal Ensembles and Tested Explanations for Performer-Independent Body-Motion Emotion Recognition
Naoto Nishida, Yoshio Ishiguro
cs.CV · cs.HC · cs.LG
Quick verdict
11-model orthogonal-error ensemble gains +11.07 pp LPO Macro-F1 with tested LMA-aligned explanations; statistically sound though modest in absolute accuracy.
Summary
The paper tackles 12-class acted-emotion recognition from body-only skeleton motion in the DIEM-A/MMAC 2026 challenge, evaluated under leave-performer-out (LPO) where chance is 8.3% and a reproduced STGCN++ baseline reaches 25.73 ± 4.03% Macro-F1. Rather than proposing new architectures, the authors ensemble eleven models spanning four inductive-bias families (graph-convolutional, attention, hybrid/MLP, frozen external pretraining) via equal-weight logit-mean fusion, reaching 36.80 ± 4.00% per-fold Macro-F1 under 10-fold LPO on the 74-performer training split — +11.07 pp (+43% relative) protocol-matched; the submitted ensemble scored 37.23% on the hidden test set, winning the Best Performance Award. The central methodological contribution is a tested explanation suite: part-masking faithfulness (+0.124 AUC gap, positive in all folds), noise stability (ρ = +0.983), counterfactual motion edits, and a region-level saliency–Laban Movement Analysis alignment (ρ = +0.500 vs +0.033 for classical kinematics; +0.517 for the full ensemble), all post hoc without retraining; the suite also reports a deliberate negative (diffuse within-window temporal saliency). Limitations: absolute accuracy near 37% remains far from deployable; models see only 64 frames (~0.53 s) of multi-second clips; data are acted, East-Asian-only; the headline LMA-vs-kinematics contrast carries wide CIs and no direct test of the difference; all members use a single training seed.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- An equal-weight logit-mean ensemble of eleven models from four inductive-bias families reaches 36.80 ± 4.00% per-fold Macro-F1 under 10-fold leave-performer-out validation, a protocol-matched +11.07 pp (+43% relative) over the same-split reproduced STGCN++ baseline (25.73 ± 4.03%).
- Part-masking and counterfactual tests demonstrate that the explained member's decisions — and the submitted ensemble's — depend on motion-grounded body-region evidence (faithfulness AUC gap +0.124 for the member; +7.85 pp part-gap for the ensemble under shuffle-masking).
- Region-level saliency aligns with rule-based Laban Movement Analysis attributes far more than with classical kinematics (ρ = +0.500 vs +0.033 for the member; ρ = +0.517 for the submitted ensemble), as a post-hoc audit requiring no retraining.
- The explanation suite faithfully reports a negative: within-window temporal saliency is diffuse (per-sample entropy 98.7% of log T; important-reverse AUC gap ≈ +0.002).
- The submitted ensemble scored 37.23% Macro-F1 on the hidden MMAC 2026 test set, winning the Best Performance Award, corroborating the cross-validated result.
Strengths
Protocol-matched comparison discipline
The headline never compares against the 92-performer official baseline (25.21 ± 4.49%, §III); all gains are against the same-split reproduction (25.73 ± 4.03%, within one SD of the official anchor), the reporting convention is restated in every caption, and the hidden-test 37.23% lands within the cross-validated spread — a consistency check most challenge papers skip.
Statistical treatment respects data structure
The 48 emotion×region LMA pairs are treated as non-independent: emotion-block bootstrap CI [+0.150, +0.733], a permutation null shuffling within emotion (p < 0.001), and an exact sign test (10/12 emotions, p = 0.039); paired-fold gain CIs [9.86, 12.33] pp survive performer-level resampling ([9.14, 12.17]) (§V, §VI-C).
Explanations are falsified, not asserted
Rare, well-documented negative-results catalog
Concrete ethics disclosure
Weaknesses
The 15× LMA-vs-kinematics headline is untested as a difference
LMA/kinematics dichotomy is partly conflated
Single training seed for all members
Abstract/body inconsistency on the human ceiling
Explanation depth concentrated on one member with non-commensurable ensemble probes
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This paper (arXiv 2609.02510v2, accepted to ACII Workshops 2026) documents the winning system of the MMAC 2026 / DIEM-A challenge: 12-class acted-emotion recognition from body-only skeleton motion under leave-performer-out (LPO) evaluation, where chance is 8.3% and the official STGCN++ baseline reaches only ~25% Macro-F1. Two claims structure the work. The performance claim: an 11-model ensemble spanning four inductive-bias families (graph-convolutional, attention, hybrid/MLP, frozen external pretraining), fused by equal-weight logit-mean, reaches 36.80 ± 4.00% per-fold Macro-F1 under 10-fold LPO on the 74-performer training split — a protocol-matched +11.07 pp (+43% relative) over the same-split reproduced baseline (25.73 ± 4.03%); the submitted system scored 37.23% on the hidden test set (Best Performance Award). The methodological claim: a 'tested explanation suite' shows the model's decisions depend on motion-grounded body-region evidence and that region saliency aligns with rule-based Laban Movement Analysis (LMA) far more than with classical kinematics (ρ = +0.500 vs +0.033 for the member; +0.517 for the submitted ensemble), while honestly returning one negative (diffuse within-window temporal saliency).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy did the MMAC 2026 body-motion emotion recognition winner achieve?
The winning system, an 11-model ensemble with equal-weight logit-mean fusion, scored 37.23% Macro-F1 (37.50% accuracy) on the hidden DIEM-A test set and received the Best Performance Award. For context: random guessing is 8.3% on the 12-way task, the official STGCN++ baseline is about 25%, and the paper's abstract cites a human score of 39%.
What does leave-performer-out (LPO) evaluation mean in emotion recognition?
Instead of splitting clips randomly, LPO holds out entire performers, so the model is always tested on people it never saw during training. This mimics real deployment on new users and is much harder: in this paper 10 folds over 74 training performers drop the reproduced baseline to about 25.7% Macro-F1 versus an 8.3% chance level.
Which body parts does the model use to recognize emotion from skeleton motion?
According to the paper's masking and counterfactual tests, head-region evidence dominates — masking the parts the model relies on degrades accuracy, a head-freeze edit is the most disruptive perturbation, and the salient body regions align with rule-based Laban Movement Analysis descriptors (ρ ≈ 0.5) far more than with raw speed or acceleration features (ρ ≈ 0.03).
Is the code or dataset for this paper publicly available?
Code is public at github.com/nawta/diema-challenge, including the deterministic regeneration script for all reported numbers. The DIEM-A motion corpus is licensed for research use via its dataset holders, but the authors' derived motion-to-rationale dataset is deliberately withheld pending performer-consent and license review.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures (S3-S5 saliency/LMA grids, S6 explanation cards) are available only as extracted text labels; visual plot content could not be inspected.
- No truncation detected: references, Tables I-III, and supplementary sections S1-S9 with captions are complete in the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.