G-Mamba: Sparse Graph-Guided Mamba for Audio-Visual Speech Enhancement
Guo-Ruei Tseng, Hung-Shin Lee, Hsin-Min Wang, Berlin Chen
eess.AS · cs.CL · cs.SD
Quick verdict
SG-Mamba: graph-guided fusion + Mamba lifts lightweight AVSE; well-ablated, but small gains lack significance tests and baseline fairness is questionable.
Summary
SG-Mamba is a lightweight audio-visual speech enhancement (AVSE) framework that replaces dense cross-attention fusion with a sparse heterogeneous graph over audio and visual frame nodes. Within a +/-3-frame window, audio-audio and visual-audio edges are processed by GATv2-style dynamic attention and visual-visual edges by GCN mean aggregation, stacked in a 4-layer U-Net (HGFF, effective receptive field +/-12 frames); a unidirectional Mamba block then models long-range context, and FCN layers predict a magnitude mask applied to the power-compressed STFT with noisy phase reuse. Built on RAVEN with frozen AVHuBERT+TalkNet visual front-ends, it is trained on 433h LRS3 mixed with DNS noise/interferers at -10 to 10 dB SNR. On LRS3 noise-only it reaches 13.091 dB SI-SDR (vs 12.648 RAVEN, 12.463 Mamba) at 5.3M trainable params / 3.45G MACs, leads all metrics under 1-interferer conditions, and stays competitive with 3 interferers; zero-shot VoxCeleb2 transfer gives the best overall average (5.567 dB SI-SDR). Ablations cover five fusion strategies and graph hyperparameters (k=3, delta=0 optimal). Main limitations: no significance testing or seed variance for small deltas, no tuning of attention baselines whose 0 dB collapse may reflect training pathologies, and compute figures exclude the frozen visual front-end.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Replacing dense cross-attention with a sparse +/-3-frame heterogeneous graph (GATv2-style attention for audio edges, GCN for visual edges) yields the best SI-SDR across noise-only, 1-interferer, and 3-interferer conditions compared with concatenation-style, FiLM, linear-attention, and cross-attention fusion at a similar parameter budget (Table IV).
- Explicit cross-frame visual-audio edges provide the largest robustness gains under severe interference, with SI-SDR +0.576 dB over RAVEN at 1-interferer 0 dB and margins shrinking as SNR rises (Sec. IV-B).
- Gains decompose into a graph-fusion component (SG-RAVEN > RAVEN by +0.209 dB) and a backbone component (SG-Mamba > SG-RAVEN by +0.234 dB), supporting the decoupling of local cross-modal alignment from global temporal modeling.
- The structural prior transfers zero-shot: on VoxCeleb2 without fine-tuning, SG-Mamba achieves the best overall average SI-SDR (5.567 dB), though plain Mamba is marginally better on interferer-only conditions (2.940 vs 2.911 dB).
Strengths
Unified, fairness-conscious baseline retraining
All baselines (Sec. III-C, Tables I-III) share audio/visual encoders and D=400; RAVEN receives an FCN projection and the same audio skip connection as SG-Mamba, and the authors state no architecture-specific hyperparameter tuning was used — reducing confounds from unequal reconstruction pipelines.
Two-axis ablation supporting the design story
Table IV compares five fusion mechanisms under matched budgets, and Table V sweeps window size k in {1,3,5} and shift delta in {-2,0,+2}; the k=3 symmetric optimum is interpreted consistently with audio-visual asynchrony literature (~+/-30 ms window, cites [26][27]).
Honest reporting of unfavorable results
Gains concentrated where visual anchoring should matter
Backbone-vs-fusion attribution via SG-RAVEN
Weaknesses
No statistical uncertainty anywhere
Attention-baseline collapse may be a training artifact
Efficiency accounting excludes the visual front-end
Mask-based regression artifacts acknowledged but unaddressed
Fixed topology and clean-visual assumptions
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is SG-Mamba and what does it do?
SG-Mamba is a lightweight audio-visual speech enhancement model: it cleans noisy speech using both the audio signal and lip-video cues. It builds a sparse graph connecting audio and visual frames within a +/-3-frame window (processed by graph attention), feeds the fused features to a Mamba state-space backbone for temporal modeling, and outputs a spectral mask. It uses only 5.3M trainable parameters and about 3.45G MACs.
How much better is SG-Mamba than RAVEN?
On an LRS3-based benchmark with noise mixed at -10 to 10 dB SNR, SG-Mamba reaches 13.091 dB SI-SDR versus 12.648 dB for a retrained RAVEN baseline (+0.443 dB), with essentially equal PESQ (2.302 vs 2.300) and slightly better ESTOI (0.842 vs 0.837). It leads on all metrics in single-interferer tests and achieves the best overall average on zero-shot VoxCeleb2 transfer (5.567 vs 4.887 dB).
Is the SG-Mamba source code available?
Yes. The paper states the source code is available at github.com/0u5gary/SG-Mamba, and the implementation section lists detailed hyperparameters (hidden size 400, 64-dim graph bottleneck, Mamba dstate 64, dconv 4, expansion factor 2, dropout 0.1, effective batch 128). Exact noise-mixing configuration details are less complete in the text.
What are the main limitations of SG-Mamba?
Its cross-modal graph uses a fixed +/-3-frame topology, which the authors show becomes less advantageous with three interfering speakers and on cross-dataset transfer; it assumes clean, visible lip inputs (no occlusion handling); it reuses the noisy phase rather than reconstructing phase; and the paper reports no statistical significance testing for its mostly small (0.1-0.6 dB) improvements.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 were available only as captions and inline text in the extraction; the architecture diagrams could not be visually verified.
- Review is based on the extracted arXiv v1 text; all six tables appear complete, but a minor header ambiguity in Table V (K vs k notation) was resolved from context.
- The code repository was not inspected; reproducibility judgments rest solely on the configuration stated in the paper.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.