Complementary rPPG-Derived and Lip-Region Frequency Cues for Talking-Face Deepfake Detection
Othmane Harraq, Tamer Aldwairi
cs.CV · cs.AI · cs.CR · eess.IV
Quick verdict
Shows rPPG and lip-DCT cues complementary across 7 Celeb-DF++ deepfake generators; honest analysis, but real/fake confounds and missing baselines temper it.
Summary
The paper examines whether rPPG-derived temporal signals (extracted by RhythmFormer) and single-frame lip-region DCT coefficients provide complementary cues for detecting talking-face deepfakes across all seven generators of Celeb-DF++, under an identity-disjoint protocol (59 subjects split 41/9/9; 2,500 fakes sampled per method). In-domain, a DCT-only MLP matches or exceeds a 1D ResNet on rPPG waveforms on six of seven methods (e.g., 0.984 vs 0.824 on FLOAT) with SadTalker the only exception (0.712 vs 0.946), and Concat fusion reaches Combined AUC 0.891 versus 0.824 and 0.827 unimodal. Under leave-one-generator-out evaluation the cues split: rPPG transfers better to AniTalker, EDTalk and SadTalker, DCT to EchoMimic, FLOAT and Real3DPortrait, and IP-LAP is near chance for both (0.554/0.534). Concat averages 0.798 but trails the better single cue on three generators (by up to 0.138 on SadTalker); an oracle reaches 0.818. Lip-region DCT beats full-face DCT on six of seven methods. The authors carefully disclaim cardiac origin for the rPPG-derived signal and disclose real/fake asymmetries (frame rate 30 vs 24-25 fps; full-length windowing of reals versus one window per fake), but run no control experiments to isolate these confounds, compare against no external detector baselines, and pool validation and test identities without significance testing.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 18 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- In-domain on Celeb-DF++ TF, lip-region DCT matches or exceeds the rPPG-derived 1D ResNet on six of seven generators (all but SadTalker, 0.712 vs 0.946), and Concat fusion reaches Combined AUC 0.891 versus 0.824 and 0.827 for the unimodal baselines.
- Under identity-disjoint leave-one-generator-out evaluation, the two cues transfer to complementary generator sets, and Concat (mean 0.798) still trails the better single cue on three of seven generators, with an oracle at 0.818.
- Lip-region DCT outperforms full-face DCT on six of seven generators (Table 3), a pattern consistent with mouth-localized synthesis artifacts.
- The rPPG-derived signal is treated strictly as an empirical forensic cue; no cardiac origin is claimed, and its discriminative content may be other temporal appearance statistics.
Strengths
Rigorous identity-disjoint LOGO design
Table 2 holds out each of the seven generators entirely with identity-disjoint training and 5 seeds, revealing the cue split (rPPG better on AniTalker/EDTalk/SadTalker; DCT better on EchoMimic/FLOAT/Real3DPortrait; IP-LAP 0.554/0.534 near chance) that in-domain Table 1 alone would mask.
Epistemic honesty about the rPPG signal
The signal is consistently labeled 'rPPG-derived' with an explicit disclaimer of cardiac origin (Sec. 3.1, following ref [5]), and Sec. 5.2 discloses the frame-rate mismatch (30 vs 24-25 fps) and the windowing asymmetry that could confound detection.
Negative results reported plainly
Region-level ablation supports the lip hypothesis
Reproducibility-oriented reporting
Weaknesses
Uncontrolled real/fake construction confounds
No external detector baselines
Thin statistical inference
Single dataset, TF forgeries only
Unusual val+test pooled evaluation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This ICASSP-style short paper — a direct follow-up to the authors' own PulseGuard preprint (ref [5]) — asks whether two lightweight visual-only cues, rPPG-derived temporal waveforms from RhythmFormer and single-frame lip-region DCT coefficients, are complementary for talking-face (TF) deepfake detection across all seven TF generators of Celeb-DF++ (AniTalker, EchoMimic, EDTalk, FLOAT, IP-LAP, Real3DPortrait, SadTalker). Four claims are made: (1) in-domain, lip DCT matches or beats the rPPG 1D ResNet on six of seven methods (Table 1; only SadTalker reverses, 0.712 vs 0.946), with Concat fusion at Combined AUC 0.891 vs 0.824/0.827 unimodal; (2) under identity-disjoint leave-one-generator-out (LOGO), the cues split — rPPG transfers better to AniTalker/EDTalk/SadTalker, DCT to EchoMimic/FLOAT/Real3DPortrait, IP-LAP near chance for both — and Concat (mean 0.798) trails the better single cue on three methods (Table 2, oracle 0.818); (3) lip DCT beats full-face DCT on six of seven (Table 3); (4) the rPPG signal is an empirical cue with no cardiac-origin claim. The claims are tightly scoped and the paper is unusually candid about what its evidence does not show.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the best AUC reported for talking-face deepfake detection in this paper?
In-domain on the seven talking-face generators of Celeb-DF++, Concat fusion of rPPG-derived waveforms and lip-region DCT reaches a combined AUC of 0.891, versus 0.824 for rPPG alone and 0.827 for lip DCT alone. Under the harder leave-one-generator-out setting, fusion averages 0.798 AUC, with an oracle that picks the best single cue per generator reaching 0.818.
Does this paper claim rPPG signals detect deepfakes because fakes lack a real heartbeat?
No. The authors explicitly call the signal 'rPPG-derived' and disclaim any cardiac origin, echoing prior cautionary work (refs 5 and 17). They treat it purely as an empirical cue whose discriminative content may be other temporal appearance statistics rather than a pulse.
What is leave-one-generator-out evaluation and why does it matter here?
Each of the seven synthesis generators is held out entirely while models train on the other six, with disjoint subject identities. It measures how detection cues transfer to unseen synthesis methods. In this paper the two cues transfer to complementary sets of generators (rPPG better on AniTalker/EDTalk/SadTalker, DCT better on EchoMimic/FLOAT/Real3DPortrait), while IP-LAP is near chance for both (0.554/0.534 AUC).
Is code available for this paper?
Yes. The authors state that code and results are available at github.com/AI-Advanced-Vision-Forensics-Lab/rppg-dct-fusion. The paper also specifies the RhythmFormer checkpoint (UBFC_cross), feature extraction details, hyperparameters, and the 5-seed evaluation protocol.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 are embedded only as layout/text fragments; the qualitative waveform and DCT-map examples (Fig. 2) could not be visually assessed.
- All evaluation is on the Celeb-DF++ TF subset; no other dataset appears in the text, so cross-dataset judgments were impossible.
- Several references are 2025-2026 preprints; citation metadata could not be independently verified without network access.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.