TAMI: Temporally Aligned, Missingness-Aware, and Interpretable Multimodal Fusion for Mental Health Assessment in Older Adults with Mild Cognitive Impairment
Merna Bibars, Bolaji Omofojoye, Allan I. Levey, Rachel Hershenberg, Gari D. Clifford, Hyeokhyen Kwon
cs.CV · cs.AI
Quick verdict
TAMI aligns multimodal interview features in time; method is sound but N=49, AUROC 0.68–0.69 and p≈0.05–0.06 alignment gains remain preliminary.
Summary
This feasibility study proposes TAMI, a multimodal fusion framework that predicts depression and anxiety risk from remote video interviews of 49 older adults with mild cognitive impairment (MCI). The method addresses three gaps in prior interview-analysis pipelines: temporal misalignment (features with different sampling supports — 1 fps facial frames, 2 s/6 s sliding acoustic and rPPG windows, word-level transcripts — are binned onto a shared 1-second timeline within question-answer segments rather than paired by sequence index), timestep-level missingness (binary availability masks distinguish true dropout from valid near-zero values), and lack of joint interpretability (Integrated Gradients aggregated into modality-, question-, and temporal-level attributions). Under participant-independent 5-fold cross-validation repeated three times, the best variants reach AUROC 0.68±0.04 for depression and 0.69±0.09 for anxiety; temporal alignment contributes the largest gain (Δ≈0.10–0.13 over sequence-index baselines), question conditioning adds no significant benefit, and the missingness mask helps mainly for anxiety, where missingness correlates with the label. Attribution concentrates on eyegaze (66% for depression) and eyegaze plus head pose (81% for anxiety), with language below 3%. An open-ended-only protocol (median 5.1 min) matches full-interview depression performance (0.67, n.s.) but drops anxiety performance significantly (0.59). Main limitations: small single-site cohort, borderline statistical significance for the headline alignment gain (p=0.05–0.06) without multiple-comparison correction, labels defined by GDS/GAD-7 questionnaires rather than clinical diagnosis, and interpretability computed on one best fold (~10 test participants).
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Fine-grained temporal binning alignment yields the largest AUROC gain (Δ≥0.1) over sequence-index-paired baselines for both depression and anxiety classification.
- Question-conditioned fusion provides no significant performance benefit once temporal alignment is in place.
- Integrated Gradients attribution concentrates on eyegaze (66%) for depression and on eyegaze plus head pose (81%) for anxiety, with language features contributing under 3%.
- Responses to the four open-ended questions alone (median 5.1 min) statistically match full-interview depression screening (AUROC 0.67 vs. 0.68) but significantly underperform for anxiety (0.59 vs. 0.69).
- Timestep-level missingness masks help mainly when missingness correlates with the label (anxiety |r|≈0.18–0.19), consistent with Che et al.'s informative-missingness framework.
Strengths
Real, well-formalized methodological gap
Temporal binning preserves each feature's temporal support (Sec. III-B2); the misalignment problem is quantified concretely — same-index features recorded up to 22 s apart (Fig. 2(a)), versus the 200 ms perceptual binding window.
Honest reporting of null and mixed results
Question conditioning is shown to add no significant gain (Q+T vs. T: p=0.28/0.60 depression, 0.64/0.20 anxiety, Table VI) and mask effects are openly reported as small and mixed (Table II, p>0.05).
Careful evaluation design
Attribution hygiene
Clinically actionable endpoint
Weaknesses
Headline claim statistically fragile
Interpretability rests on one post-hoc-selected fold
Label circularity / construct validity
No simple baseline to contextualize deep-model value
Deployment-facing gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This IEEE TAFFC-formatted feasibility study (arXiv:2608.30857) proposes TAMI, a multimodal fusion framework that screens for depression and anxiety from remote Zoom interviews of 49 older adults with Mild Cognitive Impairment (ages 73.4±8.1, 47% female, Emory CEP cohort, IRB-approved). Three claimed contributions: (1) temporal binning that aligns heterogeneous feature streams (1 fps facial, 2 s/1 s acoustic, 6 s/1 s rPPG, word-level transcripts) onto a shared 1-second timeline within question-answer segments, replacing the sequence-index pairing of prior work [26]–[28] that the authors show can displace features by up to 22 s (Fig. 2(a)); (2) per-modality-feature, per-timestep availability masks distinguishing true dropout from valid near-zero values; (3) joint modality/question/time interpretability via Integrated Gradients. Labels are GDS≥10 (21/49 positive) and GAD-7≥5 (17/48). Headline results: AUROC 0.68±0.04 (depression, Non-linear+T) and 0.69±0.09 (anxiety, Linear+T), with alignment claimed as the largest gain (Δ≥0.1).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What AUROC does TAMI achieve for depression and anxiety in older adults with MCI?
In participant-independent 5-fold cross-validation (3 repetitions) over 49 participants (48 for anxiety), TAMI's best variants reached AUROC 0.68±0.04 for depression (GDS≥10) and 0.69±0.09 for anxiety (GAD-7≥5). Temporal alignment of multimodal features provided the largest gain over misaligned baselines, though the statistical significance of that gain was borderline (p≈0.05–0.06).
How does TAMI align multimodal features with different sampling rates?
Each manually segmented question-answer interval is divided into 1-second temporal bins (1–64 per answer; bin width is widened for longer answers instead of truncating). Frame-level facial features land in the bin containing their timestamp, window-level acoustic and rPPG features are assigned to every overlapping bin, and transcript words are concatenated and re-encoded per bin. This replaces the sequence-index pairing used in prior work, which the authors show can misalign features recorded up to 22 seconds apart.
Can depression be screened using only the open-ended interview questions?
In this feasibility study, a model using only responses to four open-ended questions (median 5.1 minutes) achieved depression AUROC 0.67, statistically indistinguishable from the full interview (0.68, p=0.43). The same restriction significantly hurt anxiety detection (0.59 vs. 0.69, p≈0.02), so the authors propose the shortened protocol for depression screening only, and note it requires validation in larger samples.
Which behavioral signals drove TAMI's predictions?
Integrated Gradients attribution concentrated on eyegaze (66% for depression) and on eyegaze plus head pose (81% combined for anxiety). Language features contributed under 3%, and the question groups that defined the labels (GDS, GAD-7) were not the highest-attributed — the model instead drew on social-contact and open-ended questions throughout the interview.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures (Figs. 1-12) were available only through captions and embedded axis labels in the text extraction; the rendered figure content could not be independently inspected, so figure-based judgments rely on the authors' captions and reported numbers.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.