Sensory Precision Inference for Multimodal Arbitration under Uncertainty

Summary

The authors propose a multimodal active inference framework in which a shared latent representation and modality-specific precision variables are jointly refined by gradient descent on variational free energy (Sect. 2). A learned class-dependent prior f(z) injects top-down expectations about modality reliability into the inference. The model trains amortized encoders plus five inner-loop free-energy iterations with stop-gradient reattachment, and is evaluated on a synthetic multimodal MNIST dataset: real 28×28 digit images alongside synthetic 64-dimensional auditory and 16-dimensional tactile vectors built from Gaussians with deliberately shared class groupings. Three variants are compared — fixed precision, dynamic precision only, and the full model with the learned prior. In noise experiments (one modality corrupted with Gaussian noise up to α=1.0; 100 samples per condition), inferred precision of the corrupted modality falls while uncorrupted modalities gain weight, and the full model shows the lowest latent trajectory deviation. In incongruency trials (modalities from two different digits combined; 100 samples), the fixed-precision baseline stays near chance, both dynamic variants arbitrate above it, and the full model performs best, retaining an advantage even at Δw=0. Main limitations: entirely synthetic data, no external fusion baselines, no statistical uncertainty reporting, and static observations only.