10.25673/122858">


Proceedings of International Conference on Applied Innovation in IT
2025/12/22, Volume 13, Issue 5, pp.243-254

AI-Based Arabic Sign Language to Voice Translation System


Hussein A. Mutar, Abdul Hadi M.Alaidi, Ammar A.Kazm, Saif Ali Abd Alradha Alsaidi, Haider TH. Salim Alrikabi, Ali Khalaf Hussein, Ibtihal R. N. Alrubeei and Nabaa Ali Abd Alradha Alsaeedi


Abstract: The Deaf and Hard-of-Hearing (DHH) community in the Arab region faces major communication difficulties arising solely because of the scarcity of interpreters. Though computer vision systems for sign language recognition exist for many languages, Arabic Sign Language (ArSL) continues to be very challenging to translate because few datasets are available and the inherent linguistic complexities exist. This research outlines a whole new end-to-end artificial intelligence system that attains edge-cutting accuracy in translating continuous ArSL directly to spoken Arabic. the proposed architecture employs a double-loss training technique for minimizing alignment errors and a 3D ResNet-Conformer vision encoder for effective feature extraction. Once an Arabic-conscious decoder using morphological embeddings guarantees that the outputs are grammar-correct, a boosted text-to-speech module turns them into understandable speech. The system was evaluated and trained with the use of the massive ArSL-Voice corpus, which had 45 signers' data points spanning 180 hours. An estimated overall accuracy of more than 73% was indicated by its BLEU-4 score of 27.5 and Word Error Rate (WER) of 28.7%, demonstrating remarkable efficacy. Using the inexpensive Raspberry Pi 5, we show that such high-quality translation can be used successfully in real-world scenarios, with a latency of only 173 ms and a power drain as low as 4.1W. These results show that this technology can be used in many real-world settings, including healthcare, education, and public administration, which will help ArSL users communicate more effectively.

Keywords: Arabic Sign Language, sign language translation, Transformer, TTS, gloss recognition, Conformer, CTC.

DOI: 10.25673/122858

Download: PDF

References:

  1. O. Koller, J. Forster, and H. Ney, “Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multiple signers,” Computer Vision and Image Understanding, vol. 141, pp. 108-125, 2015.
  2. N. C. Camgoz et al., “Neural sign language translation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
  3. K. Nahar et al., “A robust model for translating Arabic sign language into spoken Arabic using deep learning,” Intelligent Automation and Soft Computing, vol. 37, no. 2, pp. 2037-2057, 2023.
  4. A. Alayed, “Machine learning and deep learning approaches for Arabic sign language recognition: A decade systematic literature review,” Sensors, vol. 24, no. 23, p. 7798, 2024.
  5. T. H. Noor et al., “Real-time Arabic sign language recognition using a hybrid deep learning model,” Sensors, vol. 24, no. 11, p. 3683, 2024.
  6. M. M. El-Gayyar, A. S. Ibrahim, and M. Wahed, “Translation from Arabic speech to Arabic sign language based on cloud computing,” Egyptian Informatics Journal, vol. 17, no. 3, pp. 295-303, 2016.
  7. M. A. Hassan, A. H. Ali, and A. A. Sabri, “Enhancing communication: Deep learning for Arabic sign language translation,” Open Engineering, vol. 14, no. 1, p. 20240025, 2024.
  8. F. M. Najib, “Sign language interpretation using machine learning and artificial intelligence,” Neural Computing and Applications, vol. 37, no. 2, pp. 841-857, 2025.
  9. V. Karanjkar et al., “A survey of sign language recognition,” International Journal of Scientific Research in Engineering and Management, vol. 7, pp. 1-11, 2023.
  10. K. Assaleh et al., “Continuous Arabic sign language recognition in user dependent mode,” Journal of Intelligent Learning Systems and Applications, vol. 2, no. 1, p. 19, 2010.
  11. A. Tharwat et al., “SIFT-based Arabic sign language recognition system,” in Afro-European Conference for Industrial Advancement: Proceedings of the First International Afro-European Conference for Industrial Advancement (AECIA 2014), Springer, 2015.
  12. S. Albanie et al., “BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues,” in European Conference on Computer Vision, Springer, 2020.
  13. A. Graves, “Connectionist temporal classification,” in Supervised Sequence Labelling with Recurrent Neural Networks, Springer, 2012, pp. 61-93.
  14. J. Kong, J. Kim, and J. Bae, “HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis,” Advances in Neural Information Processing Systems, vol. 33, pp. 17022-17033, 2020.
  15. J. Shen et al., “Natural TTS synthesis by conditioning WaveNet on mel spectrogram predictions,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2018.
  16. A. Vaswani et al., “Attention is all you need,” Advances in Neural Information Processing Systems, vol. 30, 2017.
  17. A. Stoll, M. Ziegele, and O. Quiring, “Detecting impoliteness and incivility in online discussions: Classification approaches for German user comments,” Computational Communication Research, vol. 2, no. 1, pp. 109-134, 2020.
  18. O. Koller et al., “Weakly supervised learning with multi-stream CNN-LSTM-HMMs to discover sequential parallelism in sign language videos,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 9, pp. 2306-2320, 2019.
  19. H. R. V. Joze and O. Koller, “MS-ASL: A large-scale data set and benchmark for understanding American sign language,” arXiv preprint arXiv:1812.01053, 2018.
  20. A. H. Azab et al., “Masry: A text-to-speech system for the Egyptian Arabic,” in ICINCO (2), 2023.
  21. N. Alasmari and S. Asiri, “ASLDetect: Arabic sign language detection using ResNet and U-Net like component,” Scientific Reports, vol. 15, no. 1, p. 18012, 2025.
  22. M. A. Almasre and H. Al-Nuaim, “A real-time letter recognition model for Arabic sign language using Kinect and Leap Motion Controller v2,” International Journal of Advanced Engineering, Management and Science, vol. 2, no. 5, p. 239469, 2016.
  23. N. Algethami et al., “Continuous Arabic sign language recognition models,” Sensors, vol. 25, no. 9, p. 2916, 2025.
  24. K. K. Podder et al., “Signer-independent Arabic sign language recognition system using deep learning model,” Sensors, vol. 23, no. 16, p. 7156, 2023.2022.


    HOME

       - Conference
       - Journal
       - Paper Submission to Conference
       - Paper Submission to Journal
       - Fee Payment
       - For Authors
       - For Reviewers
       - Important Dates
       - Conference Committee
       - Editorial Board
       - Reviewers
       - Last Proceeding


    PROCEEDINGS

       - Volume 13, Issue 5 (ICAIIT 2025)
       - Volume 13, Issue 4 (ICAIIT 2025)
       - Volume 13, Issue 3 (ICAIIT 2025)
       - Volume 13, Issue 2 (ICAIIT 2025)
       - Volume 13, Issue 1 (ICAIIT 2025)
       - Volume 12, Issue 2 (ICAIIT 2024)
       - Volume 12, Issue 1 (ICAIIT 2024)
       - Volume 11, Issue 2 (ICAIIT 2023)
       - Volume 11, Issue 1 (ICAIIT 2023)
       - Volume 10, Issue 1 (ICAIIT 2022)
       - Volume 9, Issue 1 (ICAIIT 2021)
       - Volume 8, Issue 1 (ICAIIT 2020)
       - Volume 7, Issue 1 (ICAIIT 2019)
       - Volume 7, Issue 2 (ICAIIT 2019)
       - Volume 6, Issue 1 (ICAIIT 2018)
       - Volume 5, Issue 1 (ICAIIT 2017)
       - Volume 4, Issue 1 (ICAIIT 2016)
       - Volume 3, Issue 1 (ICAIIT 2015)
       - Volume 2, Issue 1 (ICAIIT 2014)
       - Volume 1, Issue 1 (ICAIIT 2013)


    LAST CONFERENCE

       ICAIIT 2026
         - Photos
         - Reports

    PAST CONFERENCES

    ETHICS IN PUBLICATIONS

    ACCOMODATION

    CONTACT US

 

        

         Proceedings of the International Conference on Applied Innovations in IT by Anhalt University of Applied Sciences is licensed under CC BY-SA 4.0


                                                   This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License


           ISSN 2199-8876
           Publisher: Edition Hochschule Anhalt
           Location: Anhalt University of Applied Sciences
           Email: leiterin.hsb@hs-anhalt.de
           Phone: +49 (0) 3496 67 5611
           Address: Building 01 - Red Building, Top floor, Room 425, Bernburger Str. 55, D-06366 Köthen, Germany

        site traffic counter

Creative Commons License
Except where otherwise noted, all works and proceedings on this site is licensed under Creative Commons Attribution-ShareAlike 4.0 International License.