Hi-Singers: A Comprehensive High-Quality Dataset for Expressive Audio-Driven Singing Head Synthesis
Yichi Zhang, Hui Zhang, Guanjun Liu, Yuefeng Zou, Fengzhao Sun, Jun Yu
cs.MM · cs.AI · cs.CV · cs.SD · eess.AS
Quick verdict
Claims first large-scale 2D singing-head dataset (170h, in-the-wild); useful resource, but gains shown for one retrained model only; key thresholds unreported.
Summary
The paper introduces Hi-Singers, a curated in-the-wild video dataset for audio-driven singing head synthesis: 29,608 clips (~170 hours, 1080p, 25 fps) collected from YouTube, TikTok/Douyin and Bilibili, filtered from ~4,500 hours via automated threshold-based checks (face height, SyncNet lip-sync, displacement, Euler angles, DSL-FIQA) plus manual scoring of five quality dimensions weighted by logistic regression. The authors also release a 200-clip test benchmark balanced across Chinese/English and several genres, and propose a Beat Alignment Score (BAS) measuring synchronization between audio beats and facial-motion velocity peaks. Training Hallo on Hi-Singers improves FID (23.15 vs 23.21), FVD (181.42 vs 188.34), Sync-C (7.72 vs 7.58), BAS (0.168 vs 0.139) and MOS (3.92 vs 3.85) relative to original weights; SadTalker and V-Express official weights lag far behind on BAS (0.052 and 0.076). Limitations: only Hallo is retrained on the new data (SadTalker/V-Express serve as untrained domain-gap baselines), the test set is drawn from Hi-Singers' own held-out portion, no significance tests or variance are reported, filtering thresholds and regression weights are unspecified, the number of unique identities is not given, and no consent/licensing strategy for the scraped influencer videos is described.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 5
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Hi-Singers is the first large-scale, high-quality, in-the-wild 2D video dataset specifically tailored for singing head synthesis (29,608 clips, ~170 hours, 1080p).
- Training Hallo on Hi-Singers improves every reported metric over its original weights: FID 23.15, FVD 181.42, Sync-C 7.72, BAS 0.168, MOS 3.92.
- Models trained on speech data exhibit a severe domain gap on singing, most visibly in rhythmic alignment (SadTalker BAS 0.052, V-Express 0.076).
- The proposed Beat Alignment Score quantifies synchronization between audio beats and facial kinematic velocity peaks and detects the rhythmic drift of generic models.
Strengths
Real scale and retention discipline for a genuine gap
170 hours / 29,608 clips at 1080p in-the-wild 2D (Table 1) versus SingingHead ~27h (lab 4D) and ChorusHead ~8h (3D); 4,500h raw to 170h retained implies strict curation, with named tools (face-alignment, SyncNet, solvePnP, DSL-FIQA) at each automated stage (Sec. 4.2).
Task-specific rhythmic metric
BAS (Sec. 5.1.3), adapted from dance evaluation [13], uses Gaussian weighting (sigma=0.2, penalizing offsets beyond ~200ms) between audio beats and kinematic velocity peaks — it separates models that Sync-C cannot: SadTalker 0.052 and V-Express 0.076 vs Hallo(Hi-Singers) 0.168 (Table 3).
Balanced, multi-axis evaluation benchmark
Consistent cross-domain gains for the retrained model
Weaknesses
Architecture breadth is overclaimed
No statistical rigor
Text contradicts its own Table 4
Circular evaluation and unexplained outlier
Reproducibility and ethics gaps for a dataset paper
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Hi-Singers (ACM MM '26, arXiv:2609.22264) is a dataset paper for audio-driven singing head synthesis. The authors posit a Domain Gap between speech-trained talking-head models and singing: exaggerated expressions, vivid jaw openings, and beat-aligned dynamics, with generic models suffering rhythmic drift (Sec. 1). Four contributions: (1) the first large-scale in-the-wild 2D singing dataset — 29,608 clips, ~170 h, 1080p; (2) a 200-clip benchmark balanced across Chinese/English and five genre buckets; (3) the Beat Alignment Score (BAS) for rhythmic sync; (4) validation across 3D-coefficient (SadTalker) and diffusion (Hallo, V-Express) architectures.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the Hi-Singers dataset?
A publicly released dataset on Hugging Face of 29,608 in-the-wild video clips (about 170 hours, 1080p, 25 fps) of people singing, curated by automated plus manual filtering specifically to train and evaluate audio-driven singing-face (singing head) generation models.
How is Hi-Singers different from talking-head datasets like HDTF or VoxCeleb?
It targets singing rather than speech: singing involves larger jaw openings, more exaggerated expressions, and motion synchronized to music beats. The paper shows that models trained on talking data suffer rhythmic drift when generating singing, which the dedicated singing data mitigates.
What is BAS (Beat Alignment Score) in this paper?
A metric the authors adapt (from music-dance evaluation) to measure how well peaks in facial motion velocity align with audio beats, using Gaussian weighting with sigma 0.2 that penalizes offsets beyond roughly 200 ms; low BAS indicates rhythmic drift between the generated face and the music.
What results does training on Hi-Singers achieve?
For the Hallo model, FID 23.15, FVD 181.42, Sync-C 7.72, BAS 0.168, MOS 3.92 versus 23.21 / 188.34 / 7.58 / 0.139 / 3.85 with original weights. These gains were demonstrated for Hallo only; SadTalker and V-Express kept official weights and score far lower on BAS (0.052 and 0.076).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-6 are images in the extraction; numbers that exist only inside figures (e.g., Figure 2 Sankey retention counts, Figure 4 language/genre distribution bars) could not be verified from the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.