MultiHuSE: A Multimodal Dataset for Humour Styles and Emotions
Mary Ogbuka Kenneth, Foaad Khosmood, Abbas Edalat
cs.CL · cs.CV · cs.MM
Quick verdict
First English multimodal humour-style dataset (2,407 clips, 50 actors); valuable, but text-leakage split risk and untested fusion gains temper the claims.
Summary
MultiHuSE is a controlled-studio multimodal dataset for humour-style recognition: 2,407 HD videos (1080p, 25 fps) of 50 London-based actors performing 1,463 texts — 1,129 jokes spanning affiliative, aggressive, self-enhancing and self-deprecating styles plus 334 neutral — with performer self-reported emotions in six categories covering 2,150 instances (89.3%). Its distinguishing feature is 943 re-performances of identical texts, enabling analysis of expressive variation. Baselines using BERT, Dasheng-0.6B and MC3-18 features with XGBoost and a cross-modal attention transformer show text dominant (77.4% accuracy), exponential weighted fusion best (80.1%), with the largest per-style gain on affiliative humour (F1 66 to 74); emotion classification is weak (best 38.6%, with cross-attention below text-only). Re-performances show significantly smaller within-joke facial-feature distances than random pairs (Mann-Whitney p<0.000001, d=0.43). Key limitations: the 80:20 split is stratified only by humour style, so same-text performances may leak across train and test, inflating text-only results and the unverified +2.7% fusion gain; emotion self-report agreement is fair (κ=0.325); internal counts are inconsistent (943 vs '1,872 videos/927 jokes'); data is EULA-gated with no code release.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- MultiHuSE is the first English-language multimodal dataset annotated with psychological humour styles (the four Martin et al. categories) plus emotion labels.
- Recording multiple performances of identical texts (943 re-performances) enables systematic analysis of expressive variation in humour delivery.
- Multimodal fusion outperforms unimodal baselines for humour-style classification (80.1% exponential fusion vs 77.4% text-only accuracy), with the largest gain on affiliative humour (F1 66 to 74).
- Emotion-humour style associations (aggressive→anger/superiority, self-deprecating→embarrassment, affiliative→joy) replicate across original and re-performed takes, supporting psychological theory.
- Within-joke facial-feature distances are significantly smaller than random-pair distances (p<0.000001, d=0.43), indicating both consistency and interpretable variation across performances.
Strengths
First-of-kind dataset design
First English-language multimodal dataset annotated with the four Martin et al. humour styles plus emotion labels; 2,407 controlled 1080p/25fps studio recordings eliminate the domain noise of TED-talk (UR-FUNNY) and sitcom (MHD) predecessors compared in Table I.
Re-performances enable expression-variation research
943 re-performances (~40% of the dataset) of identical texts (Sec. III-A d); Fig. 1 contrasts anger vs superiority deliveries of the same aggressive joke, and Table I shows no compared dataset offers multiple performances of identical content.
Style-emotion structure replicates across takes
Candid limitation reporting
Concrete ethics handling
Weaknesses
Split design risks text leakage
No uncertainty quantification for the headline gain
Weak and partly contradictory emotion results
Internal numerical inconsistencies
Gated reproducibility
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
MultiHuSE (CBMI 2025) introduces a controlled-studio multimodal dataset for humour-style recognition: 2,407 1080p/25fps videos of 50 London-based actors performing 1,463 texts from the authors' prior humour-style corpus (Kenneth et al. [2]) — 1,129 jokes across the four Martin et al. styles plus 334 neutral items — with performer self-reported emotions (six categories) covering 2,150 instances (89.3%). The claimed contributions are: (1) the first English multimodal dataset with psychological humour-style plus emotion annotations; (2) multiple performances of identical texts (943 re-performances, ~40%); (3) baseline evidence that fusion beats unimodal models (80.1% vs 77.4%). The abstract frames those numbers as accuracy while contribution 3 calls them F1 — a small but telling inconsistency, since Table II lists accuracy 80.1/77.4 and F1 80/77 separately.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the MultiHuSE dataset?
MultiHuSE is a multimodal dataset of 2,407 high-definition studio videos in which 50 demographically diverse actors perform 1,463 text samples spanning four psychological humour styles (affiliative, aggressive, self-enhancing, self-deprecating) plus neutral content, with performer self-reported emotions covering 89.3% of videos. It was presented at CBMI 2025 and is available for academic use under an End-User Licence Agreement.
How big is MultiHuSE and who appears in it?
It contains 2,407 videos averaging 6.66 seconds (range 2 to 74.96 seconds) from 50 actors aged 18-69 (54% female, 40% male, 6% other genders; 72% White, 12% Asian/Asian British, 6% Black/African/Caribbean, 10% other backgrounds), recruited in London. 943 of the videos re-perform texts that already appear in the dataset.
What accuracy do models achieve on MultiHuSE?
In the paper's baselines, text-only features reach 77.4% accuracy on five-way humour-style classification, and exponential weighted multimodal fusion reaches 80.1%. Emotion classification is much harder: the best six-class result is 38.6%, and cross-attention fusion (32.2%) actually falls below text-only. No human-performance baseline is reported.
Can I download the MultiHuSE dataset?
Not openly. Researchers can request access via the project website under an End-User Licence Agreement restricted to academic research, and no baseline code is published, so replication currently requires both an approved EULA request and a reimplementation of the models.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-2 are image content not recoverable from text extraction; the style-emotion counts cited in this review (e.g., aggressive-anger 60/74, self-deprecating-embarrassment 70/55) come from the prose that quotes the heatmaps.
- Extraction garbled isolated symbols (one instance lost the £ sign: 'a 50 compensation'; kappa written out); every number cited in this review was verified as present in the provided text, which ends complete at reference [37].
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.