ReH-FUSE: Reliability-Aware Hierarchical Fusion of Experts for Multimodal Emotion Recognition in Conversation
Guan-Hua Wen, Hou-Chiang Tseng, Kuan-Yu Chen
cs.LG
Quick verdict
Proposes hierarchical text/audio/cross-modal routing for ERC; strong IEMOCAP W-F1 but gains confounded by 7B backbones and tiny, untested ablation margins.
Summary
ReH-FUSE is a method paper on multimodal emotion recognition in conversation (ERC). It runs three experts — a text expert over Qwen2.5-7B with LoRA, an audio expert over Qwen2-Audio-7B, and a bidirectional cross-attention expert — each producing class logits, and combines them with a two-stage decision-level router that first arbitrates text versus audio, then balances the unimodal mixture against the cross-modal expert. Training uses focal loss, an entropy regularizer, and a warm-up-then-joint schedule. Across three runs it reports 74.34% weighted F1 and 73.11% macro F1 on IEMOCAP, 3.44 points above the strongest listed comparator, and 68.03% weighted F1 on MELD, below MiSTER-E. Matched IEMOCAP ablations show uniform averaging (73.66) and a flat gate (74.60) trail the hierarchical router (74.94), while removing the cross-modal expert causes the largest drop (66.37); notably, a text-only inference path reaches 74.57. Class-conditional gate weights vary by emotion, and per-class F1 exposes MELD failures on fear (25.5) and disgust (37.2). The authors candidly note the 0.34-point hierarchical advantage is modest, that gate weights are proxies rather than calibrated reliability, and that large backbones confound routing gains. No code release is stated, and no variance or significance statistics accompany the three-seed results.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 3
● low 3
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A two-stage hierarchical decision router that separates text-audio preference from unimodal-vs-cross-modal selection beats a flat three-way gate by 0.34 W-F1 in matched IEMOCAP ablations (74.94 vs 74.60, three seeds).
- ReH-FUSE reaches 74.34% weighted F1 and 73.11% macro F1 on IEMOCAP over three runs — 3.44 points above the strongest listed comparator — and 68.03% weighted F1 on MELD.
- Learned routing outperforms uniform expert averaging (74.94 vs 73.66 W-F1), and removing the cross-modal expert causes the largest ablation loss (66.37).
- Routing weights vary across utterances and emotion classes rather than collapsing to fixed fusion, but they are learned proxies for trust, not calibrated measurements of modality quality.
Strengths
Matched-architecture ablation suite
Table 3 isolates routing structure from expert availability over three seeds: uniform average 73.66, flat three-way gate 74.60, hierarchical router 74.94, w/o cross-modal expert 66.37, text-only path 74.57, audio-only path 74.42, full-joint schedule 71.34 — with the single-path caveat (shared modules see the full objective) stated explicitly in §4.2.
Unusually candid self-qualification
§3.1 defines reliability as learned routing preference not calibrated signal quality; §4.1 admits initial preference may reflect logit scale; §4.3 calls gate analysis descriptive not causal; §5 lists the backbone confound and the missing corruption/missing-modality tests.
Transparent per-class and gate reporting
Useful negative results on training schedule
Weaknesses
Backbone-scale confound unquantified
No variance or significance reporting
Inconsistent headline numbers
MELD evidence asymmetric and per-class collapse
Reproducibility gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ReH-FUSE (arXiv 2609.13857, cs.LG) formulates multimodal emotion recognition in conversation as instance-dependent routing over three decision sources: a dialogue-aware text expert (Qwen2.5-7B with LoRA), an audio expert (Qwen2-Audio-7B), and a bidirectional cross-attention cross-modal expert (§3.2–3.3, Fig. 1). Its central architectural claim is that a two-stage decision-level router (§3.4) — first arbitrating text-vs-audio preference (m_i), then balancing the unimodal mixture against the cross-modal expert (f_i, Eq. 6) — separates "unimodal competition" from "cross-modal selection" and outperforms both flat three-way gating and uniform averaging. Empirical claims: 74.34% weighted F1 / 73.11% macro F1 on IEMOCAP over three runs, +3.44 over the strongest listed comparator (Table 2); 68.03% W-F1 on MELD, below MiSTER-E (69.5); learned routing beats uniform averaging; gate weights vary by emotion class (Table 4). The paper is careful from the start: §3.1 states "reliability denotes learned routing preference, not calibrated signal quality."
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ReH-FUSE?
ReH-FUSE is a multimodal emotion recognition model for conversations. It runs three predictors — a text expert built on Qwen2.5-7B, an audio expert on Qwen2-Audio-7B, and a cross-modal expert — and a two-stage router decides, for each utterance, how much each predictor contributes to the final emotion decision.
How well does ReH-FUSE perform?
Across three runs it reports 74.34% weighted F1 and 73.11% macro F1 on IEMOCAP, 3.44 points above the strongest listed prior method. On MELD it reports 68.03% weighted F1, competitive but below MiSTER-E (69.5), with very low per-class F1 on rare emotions such as fear (25.5) and disgust (37.2).
Is code for ReH-FUSE available?
The paper does not mention any code or model release. Table 1 covers backbones and input configuration, but the router temperature, text-prior bias, entropy-regularizer weight, optimizer, and LoRA settings are not specified.
Does the hierarchical router actually matter?
Only modestly: in matched ablations it beats a flat three-way gate by 0.34 weighted-F1 points and a text-only inference path by 0.37, over three seeds with no significance testing. Removing the cross-modal expert hurts far more (down to 66.37). The authors themselves call the hierarchical advantage modest.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–2 are images; only their captions and in-text descriptions are present in the extraction, so Fig. 2's exact curve values cannot be verified from the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.