An Intelligent Multimodal Framework for Identifying Children with Autism Spectrum Disorder

Early identification can significantly improve the prognosis of children with autism spectrum disorder (ASD). Yet existing identification methods are costly, time consuming, and dependent on the manual judgment of specialists. In this study, we present a multimodal framework that fuses data on a child’s eye fixation, facial expression, and cognitive level to automatically identify children with ASD, to improve the identification efficiency and reduce costs. The proposed methodology uses an optimized random forest (RF) algorithm to improve classification accuracy and then applies a hybrid fusion method based on the data source and time synchronization to ensure the reliability of the classification results. The classification accuracy of the framework was 91%, which is higher than that of the RF, support vector machine, and discriminant analysis methods. The results suggest that data on a child’s eye fixation, facial expression, and cognitive level are useful for identifying children with ASD. Because the proposed framework can separate ASD children from typically developing (TD) children, it can facilitate the early identification of ASD and may improve intervention programs for children with ASD.