Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras
Jiaqi Chen, Qinfu Xu, Hao Zhuang, Liyuan Pan
cs.CV · cs.AI
Quick verdict
Introduces 13k-clip event-camera emotion dataset and tri-modal fusion claimed SOTA; dataset valuable, but gains lean on audio/text and stats lack rigor.
Summary
The paper introduces Emo-DVS, a 13,066-clip event-camera dataset for emotion analysis, and IGF, a tri-modal fusion framework. The FAU subset (4,042 clips) was recorded with a hybrid Prophesee DVS + RGB rig under illumination from 100–150 lux down to 2–5 lux, at multiple distances and with/without glasses; the 9,024-clip emotion subset builds on posed data from VideoEmotion8 and Ekman6 plus spontaneous web videos, all converted to events via v2e simulation. IGF pre-trains a CLIP-based event encoder on FAU recognition with text-anchored contrastive objectives, then fuses event, HuBERT audio, and RoBERTa text features through temporal attention pooling, learned modality gating, and mutual-information (InfoNCE) alignment. Reported accuracy: 41.05%/56.40%/42.54% on FAU/Ekman6*/VideoEmotion8*, beating adapted baselines, and 59.43% when transferred to RGB Ekman6; LVface identification drops from 98.34% (RGB) to 32.12% (events). Key limitations: no significance testing or seed variance anywhere; the emotion subset is entirely simulated events; the compared event baselines are mostly unimodal (the paper's own event-only variant scores 29.61% on VideoEmotion8*, below NEFER's 33.95%); and code/dataset are only promised under a data use agreement.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Emo-DVS is the first large-scale event-based emotion analysis dataset, with 13,066 clips coupling dynamic illumination to hierarchical FAU and emotion annotations.
- Event cameras retain emotion-relevant signal under extreme illumination where RGB fails (37.35% vs 8.93% FAU accuracy in low light, Table 6).
- Event streams reduce state-of-the-art face identification from 98.34% (RGB) to 32.12% (Table 7), supporting hardware-level privacy preservation.
- IGF achieves state-of-the-art accuracy of 41.05%/56.40%/42.54% on FAU/Ekman6*/VideoEmotion8* and transfers to the RGB Ekman6 dataset at 59.43% (Tables 2–3).
- Pre-training on the FAU subset is a major contributor, worth roughly 8–9 accuracy points on VideoEmotion8* (Table 4).
Strengths
Largest event-based emotion dataset to date
13,066 clips versus FACEMORPHIC's 3,148 and NEFER's 609 (Table 1); the real FAU recordings span 2–5 lux to 100–150 lux, three camera distances, and glasses/no-glasses occlusion, with 29 AUs plus 8 emotion classes and 15 human participants plus 2,721 public-dataset identities.
Leakage-aware and ethics-conscious protocol
Cross-subject 70/10/20 split for the identity-sensitive FAU subset and source-disjoint splits for web videos (§3.3); institutional ethics approval, written consent, and DUA-gated access (§3.4) — above field norm.
FAU pre-training is clearly load-bearing and transfers
Each headline claim gets a dedicated experiment
Weaknesses
The SOTA margin comes from added modalities, not the event framework
Zero statistical rigor
Internal inconsistency in the central ablation
Soft ground truth and unaudited leakage channels
Simulated-event conflation in the scale and robustness claims
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (ACM MM '26) makes three coupled contributions: (1) Emo-DVS, billed as the first large-scale event-camera emotion dataset — 13,066 clips split into a 4,042-clip FAU subset (29 action units, 15 participants, aged 25–56, 2:1 male:female) recorded with a Prophesee DVS + IMX415 RGB hybrid rig under 100–150 lux down to 2–5 lux, multi-distance views, and glasses/no-glasses conditions; and a 9,024-clip emotion subset (5,442 posed from VideoEmotion8/Ekman6, 3,582 spontaneous from TikTok/YouTube/Instagram) converted to events via v2e simulation; (2) a tri-modal benchmark over event + audio + text; (3) IGF, a fusion framework with FAU pre-training (frozen CLIP spatial backbone + temporal transformer, text-anchored contrastive objective), Temporal Attention Pooling, learned modality gating, and InfoNCE-based mutual-information maximization. Headline claims: SOTA on the new benchmark (41.05%/56.40%/42.54% Acc on FAU/Ekman6*/VideoEmotion8*, Table 2), transfer to RGB Ekman6 at 59.43% (Table 3), illumination robustness (Table 6), and privacy protection (Table 7: LVface 98.34% on RGB vs 32.12% on events).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the Emo-DVS dataset?
A 13,066-clip benchmark for emotion analysis with event cameras. It combines a 4,042-clip Facial Action Unit subset recorded with a real Prophesee event sensor under lighting from about 100–150 lux down to 2–5 lux, and a 9,024-clip emotion subset (posed and spontaneous) whose videos were converted into event streams with the v2e simulator.
How privacy-preserving are event-camera face recordings according to this paper?
The LVface face recognizer identifies people in 98.34% of RGB clips but only 32.12% of event clips on the FAU subset — a drop of about 66 percentage points. That is a large reduction in identifiability, though 32% recognition is still far from full anonymity.
What accuracy does the proposed IGF method achieve?
The paper reports 41.05% accuracy on the FAU subset, 56.40% on Ekman6*, and 42.54% on VideoEmotion8*, above adapted RGB- and event-based baselines; when transferred to the original RGB Ekman6 dataset it reaches 59.43% accuracy.
Where can I download Emo-DVS or the IGF code?
The paper promises public release of the codebase and dataset access for verified academic researchers under a Data Use Agreement, but no working download link appears in the reviewed text, so availability could not be verified at review time.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The abstract's '[Code]' placeholder carries no URL in the extracted text; code availability could not be verified beyond the §3.4 promise.
- Figure 5's four subplots are extracted as interleaved number runs, so exact optimal λ/τ values could not be reliably read and are not cited in this review.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.