Multimedia Tools and Applications

, Volume 54, Issue 1, pp 181–199 | Cite as

Improving multimodal web accessibility for deaf people: sign language interpreter module

  • Matjaž Debevc
  • Primož Kosec
  • Andreas Holzinger


The World Wide Web is becoming increasingly necessary for everybody regardless of age, gender, culture, health and individual disabilities. Unfortunately, there are evidently still problems for some deaf and hard of hearing people trying to use certain web pages. These people require the translation of existing written information into their first language, which can be one of many sign languages. In previous technological solutions, the video window dominates the screen, interfering with the presentation and thereby distracting the general public, who have no need of a bilingual web site. One solution to this problem is the development of transparent sign language videos which appear on the screen on request. Therefore, we have designed and developed a system to enable the embedding of selective interactive elements into the original text in appropriate locations, which act as triggers for the video translation into sign language. When the short video clip terminates, the video window is automatically closed and the original web page is shown. In this way, the system significantly simplifies the expansion and availability of additional accessibility functions to web developers, as it preserves the original web page with the addition of a web layer of sign language video. Quantitative and qualitative evaluation has demonstrated that information presented through a transparent sign language video increases the users’ interest in the content of the material by interpreting terms, phrases or sentences, and therefore facilitates the understanding of the material and increases its usefulness for deaf people.


Human-computer interaction Usability Accessibility Deaf and hard of hearing Sign language Video Transparent video 



The project is partially supported by the European Commission within the framework of the Lifelong Learning Programme, project DEAFVOC 2. It is also partially supported by the Slovenian Research Agency in the framework of the Science to Youth program which provides financial support to young researchers. Special thanks go to Petra Rezar from the Ljubljana School for the Deaf for her comments and suggestions while using the first prototype of the application, to the Association of the Deaf and Hard-Of-Hearing People of Podravje for their help in the evaluation of the application, as well as to Milan Rotovnik from University of Maribor for his work on transparency.


  1. 1.
    Bangham J, Cox SJ, Lincoln M, Marshall I, Tutt M, Wells M (2000) Signing for the deaf using virtual humans. In: Proceedings of the IEE Seminar on Speech and Language Processing for Disabled and Elderly People (Ref. No. 2000/025), pp 4/1–4/5Google Scholar
  2. 2.
    Beskow J, Engwall O, Granström B, Nordqvist P, Wik P (2008) Visualization of speech and audio for hearing impaired persons. Technol Disabil 20(2):97–107Google Scholar
  3. 3.
    Brophy P, Craven J (2007) Web accessibility. Libr Trends 55(4):950–972Google Scholar
  4. 4.
    Cavender A, Rahul V, Barney DK, Ladner RE, Riskin EA (2007) MobileASL: intelligibility of sign language video over mobile phones. Disabil Rehabil Assist Technol 3(1):93–105CrossRefGoogle Scholar
  5. 5.
    Davis CD (1999) Visual enhancements: improving deaf students’ transition skills using multimedia technology. Career Dev Except Individ 22(2):267–281CrossRefGoogle Scholar
  6. 6.
    Debevc M, Peljhan Z (2004) The role of video technology in on-line lectures for the deaf. Disabil Rehabil 26(17):1048–1059CrossRefGoogle Scholar
  7. 7.
    Debevc M, Stjepanovič Z, Povalej P, Verlič M, Kokol P (2007) Accessible and adaptive e-learning materials: considerations for design and development. In: Universal access in human-computer interaction. Applications and Services, Lecture Notes in Computer Sciences (LNCS 4556). Springer, Berlin, Heidelberg, pp 549–558Google Scholar
  8. 8.
    EBU Technical-Information I44-2004, EBU report on Access Services—includes recommendations, Accessed 15 Sep 2009
  9. 9.
    Elliott R, Glauert J, Kennaway J, Marshall I, Safr E (2004) Linguistic modelling and language processing technologies for Avatar-based sign language presentation. Univ Access Inf Soc 6(4):375–391CrossRefGoogle Scholar
  10. 10.
    Fajardo I, Cañas JJ, Salmerón L, Abascal J (2006) Improving deaf users’ accessibility in hypertext information retrieval: are graphical interfaces useful for them? Behav Inf Technol 25(6):455–467CrossRefGoogle Scholar
  11. 11.
    Farwell RM (1976) Speechreading: a research review. Am Ann Deaf 121(1):18–22Google Scholar
  12. 12.
    Fels DI, Richards J, Hardman JL, Daniel G (2006) Sign language web pages. Am Ann Deaf 151(4):423–433CrossRefGoogle Scholar
  13. 13.
    Hanson VL (2009) Computing technologies for deaf and hard of hearing users. In: Sears A, Jacko JA (eds) Human-computer interaction handbook: fundamentals, evolving technologies and emerging applications, 2nd edn. Erlbaum, Mahwah, pp 885–893Google Scholar
  14. 14.
    Hermans D, Knoors H, Ormel E, Verhoeven L (2008) The relationship between the reading and signing skills of deaf children in bilingual education programs. J Deaf Stud Deaf Educ 13(4):518–530CrossRefGoogle Scholar
  15. 15.
    Hellström G (1997) Quality measurement on video communication for sign language. In: Proceedings of the 16th International Symposium on Human Factors in Telecommunications, Oslo, NorwayGoogle Scholar
  16. 16.
    Hilzensauer M (2006) Information technology for deaf people. In: Kacprzyk J (ed) Studies in computational intelligence. Springer, Berlin/Heidelberg, pp 183–206Google Scholar
  17. 17.
    Holley MC (2005) Keynote review: the auditory system, hearing loss and potential targets for drug development. Drug Discov Today 10(19):1269–1282CrossRefGoogle Scholar
  18. 18.
    Holt J (1995) Efficiency of screening procedures for assigning levels of the stanford achievement test (Eighth Edition) to students who are deaf or hard of hearing. Am Ann Deaf 140(1):23–27Google Scholar
  19. 19.
    Holzinger A (2005) Usability engineering for software developers. Commun ACM 48(1):71–74CrossRefGoogle Scholar
  20. 20.
    Holzinger A, Searle G, Nischelwitzer A (2007) On some aspects of improving mobile applications for the elderly. In: Constantine S (ed) Coping with diversity in universal access, Lecture Notes in Computer Science (LNCS 4554). Springer, Berlin, pp 923–932CrossRefGoogle Scholar
  21. 21.
    Huenerfauth M (2008) Generating American Sign Language animation: overcoming misconceptions and technical challenges. Univ Access Inf Soc 6(4):419–434CrossRefGoogle Scholar
  22. 22.
    Ivory M, Hearst M (2001) State of the art in automating usability evaluation of user interfaces. ACM Comput Surv 33(4):1–47CrossRefGoogle Scholar
  23. 23.
    Kennaway J, Glauert J, Zwitserlood I (2007) Providing signed content on the Internet by synthesized animation. ACM Transactions on Computer-Human Interaction 14(3)Google Scholar
  24. 24.
    Kronreif G, Dotter F, Bergmeister E, Krammer K, Hilzensauer M, Okorn I, Skant A, Orter R, Rezzonico S, Barreto B (2000) SMILE: demonstration of a cognitively oriented solution to the improvement of written language competence of deaf people. In: Proceedings of the 7th International Conference on Computers Helping People with Special Needs—ICCHP. Karlsruhe, GermanyGoogle Scholar
  25. 25.
    Marschark M, Green V, Hindmarsh G, Walker S (2000) Understanding theory of mind in children who are deaf. J Child Psychol Psychiatry Allied Discipl 41(8):1067–1073CrossRefGoogle Scholar
  26. 26.
    Martini A, Mazzoli M (1999) Achievements of the European Working Group on genetics of hearing impairment. Int J Pediatr Otorhinolaryngol 49(1):155–158CrossRefGoogle Scholar
  27. 27.
    Muir L, Richardson I (2005) Perception of sign language and its application to visual communications for deaf people. J Deaf Stud Deaf Educ 10(4):390–401CrossRefGoogle Scholar
  28. 28.
    Nigay L, Coutaz J (1993) A design space for multimodal systems: concurrent processing and data fusion. In: Proceedings of the INTERACT ’93 and CHI ’93 conference on Human factors in computing systems table of contents, Amsterdam, The Netherlands, pp.172–178Google Scholar
  29. 29.
    Olivrin GJL (2007) Is video on the web for sign languages. In: Proceedings of the W3C Video on the Web Workshop. San Jose, California and Brussels, BelgiumGoogle Scholar
  30. 30.
    Roberts VL, Fels DI (2006) Methods for inclusion: employing think aloud protocols in software usability studies with individuals who are deaf. Int J Hum Comput Stud 64(6):489–501CrossRefGoogle Scholar
  31. 31.
    Shiel B (2006) Evaluation of the social and economic costs of hearing impairment. In: Report for HEAR-IT. 2006.pdf. Accessed 15 Sep 2009
  32. 32.
    Signing Savvy (2009) Accessed 15 Sep 2009
  33. 33.
    Sign Language Interpreter Module Project (2010). Accessed 19 Feb 2010
  34. 34.
    Smith C, Mayes T (1995) Telematics applications for education and training: usability guide. Version 2, DGXIII 3/c, Commission of the European Communities. Brussels, BelgiumGoogle Scholar
  35. 35.
    Snoek CGM, Worring M (2004) Multimodal video indexing: a review of the state-of-the-art. Multimedia Tools and Applications 25(1):5–35CrossRefGoogle Scholar
  36. 36.
    Straetz K, Kaibel A, Raithel V, Specht M, Grote K, Kramer F (2004) An e-learning environment for deaf adults. In: Proceedings of the 8th ERCIM Workshop “User Interfaces for All”. Vienna, AustriaGoogle Scholar
  37. 37.
    Stewart D, Schein J, Cartwright B (1998) Sign language interpreting: exploring its art and science. Allyn & Bacon, Needham HeightsGoogle Scholar
  38. 38.
    United Nations (2006) Convention on the rights of persons with disabilities. New York: UN. Accessed 15 Sep 2009
  39. 39.
    United Nations (2006) Global Audit of Web Accessibility. New York: UN. Accessed 15 Sep 2009
  40. 40.
    Uskov V, Uskov A (2006) Web-based education: 2006–2010 perspectives. International Journal of Advanced Technology for Learning 3(3):1–13MathSciNetGoogle Scholar
  41. 41.
    Vanderheiden GC (1992) Full visual annotation of auditorially presented information for users who are deaf: ShowSounds. In: Proceedings of the RESNA International Conference. TorontoGoogle Scholar
  42. 42.
    Vogler C, Metaxas D (2001) A framework for recognizing the simultaneous aspects of American sign language. Comput Vis Image Underst 81(3):358–384MATHCrossRefGoogle Scholar
  43. 43.
    Web Content Accesssibility Guidelines (WCAG 1.0) W3C Recommendation 5 May 1999. Chisholm W, Vanderheiden G, Jacobs I (ed). Accessed 15 Sep 2009
  44. 44.
    Web Content Accesssibility Guidelines (WCAG 2.0) W3C Recommendation 11 December 2008. Caldwell B, Cooper M, Reid LG, Vanderheiden G (ed). Accessed 15 Sep 2009
  45. 45.
    World Federation of the Deaf (WFD) (2003) Position Paper regarding the United Nations Convention on the Rights of People with Disabilities. Ad Hoc Committee on a Comprehensive and Integral International Convention on the Protection and Promotion of the Rights and Dignity of Persons with Disabilities, 24 June 2003. Accessed 15 Sep 2009
  46. 46.
    YouTube (2010). Accessed 19 Feb 2010

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Matjaž Debevc
    • 1
  • Primož Kosec
    • 1
  • Andreas Holzinger
    • 2
    • 3
  1. 1.Faculty of Electrical Engineering and Computer ScienceUniversity of MariborMariborSlovenia
  2. 2.Institute of Medical Informatics, Research Unit HCI4MEDMedical University GrazGrazAustria
  3. 3.Institute for Information Systems and Computer MediaGraz University of TechnologyGrazAustria

Personalised recommendations