Advertisement

Universal Access in the Information Society

, Volume 18, Issue 4, pp 939–951 | Cite as

Automatic translation of Arabic text-to-Arabic sign language

  • Hamzah LuqmanEmail author
  • Sabri A. Mahmoud
Long Paper
  • 283 Downloads

Abstract

Arabic sign language (ArSL) is a full natural language that is used by the deaf in Arab countries to communicate in their community. Unfamiliarity with this language increases the isolation of deaf people from society. This language has a different structure, word order, and lexicon than Arabic. The translation between ArSL and Arabic is a complete machine translation challenge, because the two languages have different structures and grammars. In this work, we propose a rule-based machine translation system to translate Arabic text into ArSL. The proposed system performs a morphological, syntactic, and semantic analysis on an Arabic sentence to translate it into a sentence with the grammar and structure of ArSL. To transcribe ArSL, we propose a gloss system that can be used to represent ArSL. In addition, we develop a parallel corpus in the health domain, which consists of 600 sentences, and will be freely available for researchers. We evaluate our translation system on this corpus and find that our translation system provides an accurate translation for more than 80% of the translated sentences.

Keywords

Arabic sign language Machine translation Rule-based machine translation Arabic sign language corpus Arabic gloss system 

Notes

Acknowledgements

The authors would like to thank the referees for their constructive comments. We thank Dr. Nizar Habash for his helpful conversations, resources and feedback. We also like to thank Manal Al-Ashwal, Dr. Sameer Semreen, and Ayman Al-Qadsi who actively participated in this work. In addition, we would like to acknowledge the support provided by King Fahd University of Petroleum & Minerals (KFUPM) for funding this work through project number IN151008.

References

  1. 1.
    Tolba, M.F., Elons, A.S.: Recent developments in sign language recognition systems. In: Computer Engineering and Systems (ICCES), 2013 8th International Conference on, IEEE, 2013Google Scholar
  2. 2.
    Sidig, A.A.I., Luqman, H., Mahmoud, S.A.: Transform-based Arabic sign language recognition. Proc. Comput. Sci. 117, 2–9 (2017).  https://doi.org/10.1016/j.procs.2017.10.087 CrossRefGoogle Scholar
  3. 3.
    Al-Fityani, K., Padden, C.: Sign language geography in the Arab world, sign languages. In: Brentari, D. (ed.) Sign languages: a Cambridge survey. Cambridge University Press, New York, pp. 433–450 (2010)CrossRefGoogle Scholar
  4. 4.
    LAS: First part of the Unified Arabic Sign Dictionary. The League of Arab States & the Arab League Educational, Cultural and Scientific Organization, Tunisia (2000) (in Arabic)Google Scholar
  5. 5.
    Mohandes, M., Deriche, M., Liu, J.: Image-based and sensor-based approaches to Arabic sign language recognition. IEEE Trans. Hum. Mach. Syst. 44(4), 551–557 (2014)CrossRefGoogle Scholar
  6. 6.
    Almohimeed, A., Wald, M., Damper, R.: Arabic text to Arabic sign language translation system for the deaf and hearing-impaired community. In: Proceedings of the Second Workshop on Speech and Language Processing for Assistive Technologies, Association for Computational Linguistics, pp. 101–109 (2011)Google Scholar
  7. 7.
    Sidig, A.A.I., Luqman, H., Mahmoud, S.A.: Arabic sign language recognition using optical flow-based features and HMM. In: Recent Trends in Information and Communication Technology, pp. 297–305. Springer International Publishing, Cham (2018)Google Scholar
  8. 8.
    Zhao, L., Kipper, K., Schuler, W., Vogler, C., Badler, N., Palmer, M.: A machine translation system from English to American sign language. In: Conference of the Association for Machine Translation in the Americas, pp. 54–67. Springer, Berlin (2000)CrossRefGoogle Scholar
  9. 9.
    Marshall, I., Sáfár, É.: A prototype text to British sign language (BSL) translation system. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, vol. 2, pp. 113–116 (2003)Google Scholar
  10. 10.
    Morrissey, S., Way, A.: An example-based approach to translating sign language. In: Proceedings of the workshop in example-based machine translation (MT Summit X), Phuket, Thailand, pp. 109–116 (2005)Google Scholar
  11. 11.
    Cox, S., Lincoln, M., Tryggvason, J., Nakisa, M., Wells, M., Tutt, M., Abbott, S.: The development and evaluation of a speech-to-sign translation system to assist transactions. Int. J. Hum. Comput. Interact. 16(2), 141–161 (2003)CrossRefGoogle Scholar
  12. 12.
    Veale, T., Conway, A., Collins, B.: The challenges of cross-modal translation: English-to-sign-language translation in the zardoz system. Mach. Transl. 13(1), 81–106 (1998)CrossRefGoogle Scholar
  13. 13.
    d’Armond, L.S.: Representation of American sign language for machine translation, Ph.D. thesis, Georgetown University (2002)Google Scholar
  14. 14.
    Marshall, I., Sáfár, É.: Extraction of semantic representations from syntactic SMU link grammar linkages. In: Proceedings of Recent Advances in Natural Language Processing, pp. 154–159 (2001)Google Scholar
  15. 15.
    Grieve-Smith, A.B.: English to American sign language machine translation of weather reports. In: Proceedings of the Second High Desert Student Conference in Linguistics (HDSL2), Albuquerque, NM, pp. 23–30 (1999)Google Scholar
  16. 16.
    Van Zijl, L., Combrink, A.: The South African sign language machine translation project: issues on non-manual sign generation. In: Proceedings of the 2006 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries, South African Institute for Computer Scientists and Information Technologists, pp. 127–134 (2006)Google Scholar
  17. 17.
    San-Segundo, R., Montero, J.M., Macías-Guarasa, J., Córdoba, R., Ferreiros, J., Pardo, J.M.: Proposing a speech to gesture translation architecture for spanish deaf people. J. Vis. Lang. Comput. 19(5), 523–538 (2008)CrossRefGoogle Scholar
  18. 18.
    Huenerfauth, M., Marcus, M., Palmer, M.: Generating American sign language classifier predicates for English-to-ASL machine translation, Ph.D. thesis, University of Pennsylvania (2006)Google Scholar
  19. 19.
    Othman, A., Jemni, M.: Statistical sign language machine translation: from English written text to American sign language gloss. Int. J. Comput. Sci. Issues 8(5), 65–73 (2011)Google Scholar
  20. 20.
    Bonham, M.E.: English to ASL Gloss Machine Translation. M. Art thesis, Brigham Young University (2015)Google Scholar
  21. 21.
    Almasoud, A.M., Al-Khalifa, H.S.: Semsignwriting: a proposed semantic system for Arabic text-to-signwriting translation. J. Softw. Eng. Appl. 5, 604–612 (2012)CrossRefGoogle Scholar
  22. 22.
    El, A.E.E., Basuony, E.M.M.R., Atawy, E.S.M.: Intelligent Arabic text to Arabic sign language translation for easy deaf communication. Int. J. Comput. Appl. 92(8), 22–29 (2014)Google Scholar
  23. 23.
    Al-Rikabi, S., Hafner, V.: A humanoid robot as a translator from text to sign language. In: 5th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics (LTC 2011), pp. 375–379 (2011)Google Scholar
  24. 24.
    Mohandes, M.: Automatic translation of Arabic text to Arabic sign language. AIML J. 6(4), 15–19 (2006)Google Scholar
  25. 25.
    Halawani, S.M.: Arabic sign language translation system on mobile devices. IJCSNS Int. J. Comput. Sci. Netw. Secur. 8(1), 251–256 (2008)Google Scholar
  26. 26.
    Al-Khalifa, H.S.: Introducing Arabic sign language for mobile phones. In: International Conference on Computers for Handicapped Persons, Springer, New York, pp. 213–220 (2010)Google Scholar
  27. 27.
    Al Ameiri, F., Zemerly, M.J., Al Marzouqi, M.: M-learning and chatting using indexed Arabic sign language. Int. J. Infonomics (IJI) 5, 10 (2012)Google Scholar
  28. 28.
    Al-Nafjan, A., Al-Arifi, B., Al-Wabil, A.: Design and development of an educational Arabic sign language mobile application: collective impact with Tawasol. In: International Conference on Universal Access in Human-Computer Interaction, Springer, New York, pp. 319–326 (2015)CrossRefGoogle Scholar
  29. 29.
    Abdel-Fattah, M.A.: Arabic sign language: a perspective. J. Deaf Stud. Deaf Educ. 10(2), 212–221 (2005)CrossRefGoogle Scholar
  30. 30.
    Semreen, S., Albinali, M.: The rules of Arab Qatari sign standardized language. Supreme Council of Family Affairs, Qatar (2010) (in Arabic)Google Scholar
  31. 31.
    Habash, N., Soudi, A., Buckwalter, T.: On Arabic transliteration. In: Arabic computational morphology, Springer, New York, pp. 15–22 (2007)Google Scholar
  32. 32.
    Habash, N., Faraj, R., Roth, R.: Syntactic annotation in the columbia Arabic treebank. In: Proceedings of MEDAR International Conference on Arabic Language Resources and Tools, Cairo, Egypt (2009)Google Scholar
  33. 33.
    Filhol, M., Hadjadj, M.N., Testu, B.: A rule triggering system for automatic text-to-sign translation. Univers. Access Inf. Soc. 15(4), 487–498 (2016)CrossRefGoogle Scholar
  34. 34.
    Morrissey, S.: Data-driven machine translation for sign languages, Ph.D. thesis, Dublin City University (2008)Google Scholar
  35. 35.
    Almohimeed, A., Wald, M., Damper, R.: An Arabic sign language corpus for instructional language in school. In: LREC 2010: 4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, pp. 81–82 (2010)Google Scholar
  36. 36.
    Assaleh, K., Shanableh, T., Fanaswala, M., Amin, F., Bajaj, H.: Continuous Arabic sign language recognition in user dependent mode. J. Intell. Learn. Syst. Appl. 2(01), 19–27 (2010)Google Scholar
  37. 37.
    Elhadj, Y.O.M., Zemirli, Z., Ayyadi, K.: Development of a bilingual parallel corpus of Arabic and Saudi sign language: Part i. In: Intelligent Informatics, Springer, New York, pp. 285–295 (2013)Google Scholar
  38. 38.
    Zahedi, M., Dreuw, P., Rybach, D., Deselaers, T., Ney, H.: Continuous sign language recognition-approaches from speech recognition and available data resources. In: Second Workshop on the Representation and Processing of Sign Languages: Lexicographic Matters and Didactic Scenarios, pp. 21–24 (2006)Google Scholar
  39. 39.
    Dreuw, P., Forster, J., Deselaers, T., Ney, H.: Efficient approximations to model-based joint tracking and recognition of continuous sign language. In: Automatic Face and Gesture Recognition, 2008. FG’08. 8th IEEE International Conference on, IEEE, pp. 1–6 (2008)Google Scholar
  40. 40.
    Dreuw, P., Neidle, C., Athitsos, V., Sclaroff, S., Ney, H.: Benchmark databases for video-based automatic sign language recognition. In: Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08) (2008)Google Scholar
  41. 41.
    Athitsos, V., Neidle, C., Sclaroff, S., Nash, J., Stefan, A., Thangali, A., Wang, H., Yuan, Q.: Large lexicon project: American sign language video corpus and sign language indexing/retrieval algorithms. In: Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies (CSLT), pp. 11–14 (2010)Google Scholar
  42. 42.
    Schembri, A., Fenlon, J., Rentelis, R., Cormier, K.: British sign language corpus project: a corpus of digital video data of British sign language 2008–2010, 1st edn. University College London. http://www.bslcorpusproject.org (2011). Accessed 13 June 2018
  43. 43.
    Crasborn, O.A., Zwitserlood, I.: The corpus NGT: an online corpus for professionals and laymen. In: 3rd Workshop on the Representation and Processing of Sign Languages (LREC), ELDA, pp. 44–49 (2008)Google Scholar
  44. 44.
    Leeson, L., Saeed, J., Byrne-Dunne, D., Macduff, A., Leonard, C.: Moving heads and moving hands: developing a digital corpus of Irish sign language. The ’signs of Ireland’ corpus development project. In: IT&T conference, Carlow (2006)Google Scholar
  45. 45.
    Chiu, Y.-H., Wu, C.-H., Su, H.-Y., Cheng, C.-J.: Joint optimization of word alignment and epenthesis generation for Chinese to Taiwanese sign synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 29(1), 28–39 (2007)CrossRefGoogle Scholar
  46. 46.
    Bungeroth, J., Stein, D., Dreuw, P., Zahedi, M., Ney, H.: A german sign language corpus of the domain weather report. In: Fifth International Conference on Language Resources and Evaluation, pp. 2000–2003 (2006)Google Scholar
  47. 47.
    Stokoe, W.C.: Sign language structure: an outline of the visual communication systems of the American deaf. J. Deaf Stud. Deaf Educ. 10(1), 3–37 (2005)CrossRefGoogle Scholar
  48. 48.
    Bentele, S.: About the HamNoSys system. http://www.signwriting.org/forums/ linguistics/ling007.html. Accessed Apr 2017
  49. 49.
    Aouiti, N., Jemni, M., Semreen, S.: Arab gloss annotation system for Arabic sign language. In: Information and Communication Technology and Accessibility (ICTA), 2015 5th International Conference on, IEEE, pp. 1–6 (2015)Google Scholar
  50. 50.
    Liddell, S.K.: Grammar, gesture, and meaning in American Sign Language. Cambridge University Press, Cambridge (2003)CrossRefGoogle Scholar
  51. 51.
    Habash, N.Y.: Introduction to Arabic natural language processing. Synth. Lect. Hum. Lang. Technol. 3(1), 1–187 (2010)CrossRefGoogle Scholar
  52. 52.
    Pasha, A., Al-Badrashiny, M., Diab, M.T., El Kholy, A., Eskander, R., Habash, N., Pooleery, M., Rambow, O., Roth, R.: Madamira: a fast, comprehensive tool for morphological analysis and disambiguation of arabic., In: LREC, vol. 14, pp. 1094–1101 (2014)Google Scholar
  53. 53.
    Shahrour, A., Khalifa, S., Taji, D., Habash, N.: Camelparser: a system for Arabic syntactic analysis and morphological disambiguation. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations, pp. 228–232 (2016)Google Scholar
  54. 54.
    Nivre, J., Hall, J., Nilsson, J., Chanev, A., Eryigit, G., Kübler, S., Marinov, S., Marsi, E.: Maltparser: a language-independent system for data-driven dependency parsing. Nat Lang Eng 13(02), 95–135 (2007)CrossRefGoogle Scholar
  55. 55.
    Elkateb, S., Black, W., Rodríguez, H., Alkhalifa, M., Vossen, P., Pease, A., Fellbaum, C.: Building a wordnet for Arabic. In: Proceedings of The fifth international conference on Language Resources and Evaluation (LREC 2006) (2006)Google Scholar
  56. 56.
    Heafield, K.: Kenlm: faster and smaller language model queries. In: Proceedings of the Sixth Workshop on Statistical Machine Translation, Association for Computational Linguistics, pp. 187–197 (2011)Google Scholar
  57. 57.
    Al-Jefri, M.: Real-word error detection and correction in Arabic text, Master’s thesis, King Fahd University of Petroleum and Minerals (2013)Google Scholar
  58. 58.
    Stolcke, A.: SRILM-an extensible language modeling toolkit. In: Proceedings International Conference on Spoken Language Processing, pp. 257–286 (2002)Google Scholar
  59. 59.
    Federico, M., Bertoldi, N., Cettolo, M.: IRSTLM: an open source toolkit for handling large scale language models. In: Interspeech, pp. 1618–1621 (2008)Google Scholar
  60. 60.
    Arabic sign language dictionary. http://www.menasy.com/. Accessed Apr 2017
  61. 61.
    Marton, Y., Habash, N., Rambow, O.: Dependency parsing of Modern Standard Arabic with lexical and inflectional features. Comput. Linguist. 39(1), 161–194 (2013)CrossRefGoogle Scholar
  62. 62.
    Hadla, L.S., Hailat, T.M., Al-Kabi, M.N.: Evaluating Arabic to english machine translation. Int. J. Adv. Comput. Sci. Appl. (IJACSA) 5(11), 68–73 (2014)Google Scholar
  63. 63.
    Gonàlez, M., Giménez, J., Màrquez, L.: A Graphical interface for MT evaluation and error analysis. In: The 50th Annual Meeting of the Association for Computational Linguistics (2012)Google Scholar
  64. 64.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting on association for computational linguistics, Association for Computational Linguistics, pp. 311–318 (2002)Google Scholar
  65. 65.
    Nießen, S., Och, F.J., Leusch, G., Ney, H., Informatik, L.F.: An evaluation tool for machine translation: fast evaluation for MT research. In: In Proceedings of the 2nd International Conference on Language Resources and Evaluation (LREC-2000) (2000)Google Scholar
  66. 66.
    Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of association for machine translation in the Americas, vol. 200 (2006)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.King Fahd University of Petroleum and MineralsDhahranSaudi Arabia

Personalised recommendations