Detecting Agitation Before Behavioral Escalation in Autistic Youth Through Multimodal Wearable Sensing

Summary

This empirical study presents a multimodal wearable system for detecting agitation before behavioral escalation in 15 autistic youth. Across 30 clinician-led modified-IISCA sessions, the authors recorded upper-body movement (five shirt-embedded IMUs), wrist physiology (EmotiBit EDA/PPG, worn by 7 of 15), and lapel-microphone audio, labeled post hoc by clinicians with inter-observer agreement above 0.85. Four pretrained foundation models (UniMTS, EfficientAT, PaPaGei, UME), one per modality, are projected to a shared 128-dimensional space and fused into a single group model with per-participant heads. Under five-fold chronological block cross-validation, detection at the annotated onset reached AUC 0.724 (95% CI 0.650–0.794; permutation p=0.0005), declining to 0.608 at 30 s before onset; 13 of 15 participants exceeded chance. From-scratch training reached only 0.583, while frozen and LoRA fine-tuned features were comparable (0.708/0.724). Audio dominated: removing it cost 0.126 AUC while other modalities cost under 0.02, and a watch-only configuration stayed near chance (0.560). Key limitations are the small single-site cohort, incomplete physiology coverage, the inability to verify pre-onset labels against temporal autocorrelation, and absence of a code or data release.