An Efficient Bidirectional Android Translation Prototype for Yemeni Sign Language Using Fuzzy Logic and CNN Transfer Learning Models

Hearing-impaired individuals face significant challenges in social interactions due to communication barriers. Advances in technology have introduced numerous assistive tools to bridge this gap. This research aims to enhance communication between hearing-impaired and hearing individuals by developing a translation system for Yemeni Arabic Sign Language. The prototype leverages CNN deep learning models and fuzzy string matching to translate signs and text efficiently. The Yemeni Sign Language dataset comprises 24,245 images representing 32 commonly used words. The prototype consists of two primary modules for translating text to sign language and vice versa in both directions. The initial prototype module was created to convert sign language input from hearing-impaired individuals into text via five CNN transfer learning models: MobileNet, GoogleNet, VGG16, ResNet152, and DenseNet161. Fuzzy string-matching and data sign dictionary approaches were utilized to convert the input words into sign images. The proposed system achieved high translation accuracy rates for various CNN models, with ResNet152 scoring 98.78%, MobileNet scoring 97.94%, GoogleNet scoring 98.36%, VGG16 scoring 90.46%, and DenseNet161 scoring 98.34% based on the experimental results. Furthermore, the experimental data demonstrated that fuzzy matching score models can effectively convert the input word into a hand sign image with excellent performance. The fuzzy matching score model efficiently solves the issues of synonym words and spelling typo errors and provides a fast translation approach. The suggested model demonstrated the ability to create a bidirectional sign language translation Android application for Yemeni Arabic sign language with outstanding accuracy and performance. It also has the capacity to create a robust bidirectional Arabic sign language translation system through the utilization of deep learning techniques.