A multimodal large language model for evidence-based autism spectrum disorder screening
Jun Chen, Qi Zhao, Yunliang Jiang, Shuqin Cao, Yunqiang Lin, Chenglong Jia, Qiang Guo, Guang Dai, et al.
cs.CV · cs.HC · cs.LG
Quick verdict
Multimodal LLM for ASD screening: strong AUCs (0.953 CV, 0.932 held-out sites), but subtyping, label provenance, and reproducibility need scrutiny.
Summary
The authors present ASDchat, a multimodal large language model (fine-tuned Qwen3-VL-8B) for evidence-based autism spectrum disorder (ASD) screening from structured interaction videos, audio, and dialogue statistics. A dual-branch architecture couples a decision head that outputs a screening probability with an evidence branch that generates timestamped behavioral evidence aligned to ADOS-2 items and answers bidirectional spatio-temporal questions; site-adversarial training, evidence supervision, and a grayscale-input faithfulness loss support robustness and grounding. The model was trained on 1,035 children (370 typically developing, 350 ASD, 315 with other neurodevelopmental disorders) from 27 Chinese sites, ages 3-18. In 5-fold cross-validation, ASD versus TD reached AUC 0.953+/-0.021; ASD versus other disorders dropped to 0.719. On 9 fully held-out sites the mean AUC was 0.932, with large per-site variation (0.796-0.991). Ablations show video dominates (0.948 alone) and removing the evidence branch lowers AUC to 0.888. K-means clustering on seven behavioral dimensions yields six proposed ASD subtypes with intervention mappings. Key limitations: diagnostic ground-truth procedures are unspecified, raising circularity concerns; the subtyping lacks replication; no comparison with clinical instruments or published models on the same cohort; no code or data release; and the 3-18 age range excludes the under-3 population that motivates the work.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 4
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- ASDchat screens ASD versus typically developing children with AUC 0.953+/-0.021 in 5-fold cross-validation and 0.932 mean AUC on 9 fully held-out sites.
- The evidence branch produces timestamped, ADOS-2-aligned, verifiable behavioral evidence and improves AUC from 0.888 (without it) to 0.953.
- Unsupervised K-means clustering of seven video-derived behavioral dimensions separates 350 ASD cases into six phenotypically distinct subtypes, each mapped to intervention suggestions.
- Combining video, audio, and dialogue modalities outperforms any single modality, with visual behavior as the dominant signal.
Strengths
Clinically realistic benchmark design
Beyond the routine ASD-vs-TD task (AUC 0.953), the Comprehensive Quantitative Analysis section evaluates ASD versus non-ASD (0.861) and versus other disorders (0.719+/-0.043), and the Discussion explicitly contrasts this with prior ASD-vs-TD-only work; the hard-task degradation is reported honestly rather than hidden.
Site-level generalization protocol
The Evaluation/Cross-site section holds out 9 of 27 sites entirely, retrains the full pipeline, and reports per-site AUCs (0.796-0.991) including failure modes (Site i precision 0.571, F1 0.644), giving a transparent picture of distribution shift rather than a single pooled number.
Grounded evidence with counterfactual checks
Informative ablations
Multi-site pediatric dataset with differential groups
Weaknesses
Label provenance and potential circularity
Subtyping claim outruns the evidence
No head-to-head baselines
Reproducibility not supported
Scope mismatch with stated motivation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ASDchat and what does it do?
ASDchat is a multimodal large language model (built on Qwen3-VL-8B) that screens children for autism spectrum disorder from a 5-10 minute recorded caregiver or examiner interaction. It outputs a screening probability plus timestamped behavioral evidence aligned to ADOS-2 criteria, so clinicians can verify each judgment directly against the video.
How accurate is ASDchat for autism screening?
In the paper, separating ASD from typically developing children achieved AUC 0.953 in 5-fold cross-validation and a mean AUC of 0.932 on 9 clinical sites fully held out from training (per-site range 0.796-0.991). Distinguishing ASD from other neurodevelopmental disorders was much harder, with AUC 0.719.
Can ASDchat replace a clinical autism diagnosis?
No. It is positioned as a screening aid, not a diagnostic tool. The authors report it struggles to separate ASD from conditions like intellectual disability or ADHD (AUC 0.719), and its subtype-based intervention suggestions are not clinically validated. Formal diagnosis still requires specialist assessment.
Is the ASDchat code or dataset publicly available?
The paper states no code, model, or data release. Because the dataset contains videos of 1,035 minors from 27 Chinese clinical sites, open release is unlikely without a privacy-controlled mechanism, which the paper does not describe, so the results are currently hard to independently verify.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Most sensitivity, specificity, precision, and F1 values appear only in Figures 3-9, which could not be analyzed from the extracted text; quantitative review is limited to in-text numbers.
- Figure panels and captions (e.g., Figure 1b/2b evidence examples) were partially garbled in extraction, so some apparent inconsistencies (such as out-of-range timestamps) may be extraction artifacts rather than paper errors.
- Supplementary Table 1 (full ADOS-2 item list) and any supplementary methods were not included in the provided text.
- This review covers arXiv v1 only and does not verify claims against external sources or the underlying data.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.