Multimodal Data Fusion Framework for Early Prediction of Autism Spectrum Disorder

Autism spectrum disorder (ASD) is a condition that impacts a person’s emotional, cognitive, social, and physical well-being. Symptoms include challenges in communicating, struggles with social interactions, fixation, and repetitive actions. It is crucial to detect ASD in young children to minimize the impact of the disorder through various therapies focused on behavior, education, and family. The application of artificial intelligence has been important in detecting ASD in children. Previous studies have proposed different methods for identifying ASD, mainly using either demographic information or visual characteristics separately, without effectively combining both approaches. Our study presents a new approach to detecting ASD that takes into account both demographic and visual information. Therefore, a framework was suggested to assess different deep learning models for the early identification of ASD. The proposed framework consists of four modules such as stacked bidirectional long short-term memory (SBiLSTM) using attention mechanism for representing text/numerical features, multilevel 2D-convolutional neural network–gated recurrent units (ABM-2D-CNN–GRUs) using attention mechanism for extracting facial features, and multimodal factorized bilinear (MFB) pooling for combining the features. Moreover, the conditional probability approach calculates a distinct weight for each class based on specific features, leading to enhanced system performance. In conclusion, the AlexNet CNN has been proposed for prediction and its performance was assessed using the multiactivation function (MAF) framework. In this study, we examined the dataset for screening ASD and the dataset for children with autism. It is crucial to detect ASD at an early stage. We have identified features that can differentiate children with ASD from those without ASD. The suggested system achieves a higher accuracy rate of 99.2% compared to current systems. This outcome indicates that our system is better at predicting ASD compared to other advanced methods.