Skip to main content
SpringerLink
Log in

GIST Descriptors for Sign Language Recognition: An Approach Based on Symbolic Representation

  • Conference paper
  • First Online:
Mining Intelligence and Knowledge Exploration (MIKE 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9468))

Abstract

This paper presents an approach for recognizing signs made by hearing impaired people at sentence level. The signs are captured in the form of video and each frame is processed to efficiently extract sign information to model the sign and recognize instances of new test signs. Low-dimensional global “gist” descriptors are used to capture sign information from every frame of a sign video. K-means clustering is used to choose fixed number of frames, which are discriminative enough to distinguish between signs. Also, selection of fixed number of frames helps us to deal with unequal number of frames among the instances of same sign due to different signers and reduce the complexity of subsequent processing. Further, we exploit the concept of symbolic data analysis to effectively represent a sign. A fuzzy trapezoidal membership function is used to establish the similarity between test and a reference sign and a nearest neighbour classification technique is used to recognize the given test sign. A considerably large database of signs (UoM-ISL) is created and an extensive experimentation is conducted on this database to study the efficacy of the proposed methodology. The experimental results are found to be encouraging.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Al-Ahdal, M., Tahir, N.: Review in sign language recognition systems. In: IEEE Symposium on Computers and Informatics (ISCI), pp. 52–57 (2012)

    Google Scholar 

  2. Ghotkar, A.S., Kharate, G.K.: Study of vision based hand gesture recognition using Indian sign language. IJSS Intell. Syst. 7(1), 96–115 (2014)

    Google Scholar 

  3. Oliva, A., Torralba, A.: Building the gist of a scene: the role of global image features in recognition. In: Martinez-Conde, S., Macknik, S.L., Martinez, L.M., Alonso, J.-M., Tse, P.U. (eds.) Progress in Brain Research, vol. 155 (2006). ISSN 0079-6123

    Google Scholar 

  4. Bock, H.H., Diday, E.: Analysis of Symbolic Data. Springer, Berlin (2000)

    Book  Google Scholar 

  5. Dahmani, D., Larabi, S.: User-independent system for sign language finger spelling recognition. J. Vis. Commun. Image Represent. 25(5), 1240–1250 (2014)

    Article  Google Scholar 

  6. Gourley, C.: Neural network utilizing posture input for sign language recognition. Technical report Computer Vision and Robotics Research Laboratory, University of Tenessee Knoxville, November 1994

    Google Scholar 

  7. Gowda, K.C., Diday, E.: Symbolic clustering using a new dissimilarity measure. Pattern Recogn. 24(6), 567–578 (1991)

    Article  Google Scholar 

  8. Gowda, K.C., Ravi, T.V.: Divisive clustering of symbolic objects using the concepts of both similarity and dissimilarity. Pattern Recogn. 28(8), 1277–1282 (1995)

    Article  Google Scholar 

  9. Guru, D.S., Prakash, H.N.: Online signature verification and recognition: an approach based on symbolic representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(6), 1059–1073 (2009)

    Article  Google Scholar 

  10. Guru, D.S., Nagendraswamy, H.S.: Clustering of interval-valued symbolic patterns based on mutual similarity value and the concept of k-mutual nearest neighborhood. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3852, pp. 234–243. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  11. Guru, D.S., Kiranagi, B.B., Nagabhushan, P.: Multivalued type proximity measure and concept of mutual similarity value useful for clustering symbolic patterns. Pattern Recogn. Lett. 25(10), 1203–1213 (2004)

    Article  Google Scholar 

  12. Handouyahia, M., Zion, D., Wang, S.: Sign language recognition using moment based size functions. In: Vision Interface 1999, Trois-Rivieres, Canada, 19–21 May 1999

    Google Scholar 

  13. Handouyahia, M.: Sign Language Recognition using moment based size functions. MSc en Informatique demathematique et d informations, universite de sherbrooke, sherbrooke (1998)

    Google Scholar 

  14. Hu, M.: Visual pattern recognition by moment invariants. IRE Trans Inf. Theory 8, 179 (1962)

    MATH  Google Scholar 

  15. Kang, S., Nam, M., Rhee, P.: Colour based hand and finger detection technology for user interaction. In: International Conference on Convergence and Hybrid Information Technology, pp. 229–236 (2008)

    Google Scholar 

  16. Karthick, P., Prathiba, N., Rekha, V.B., Thanalaxmi, S.: Transforming Indian sign language into text using leap motion. Int. J. Innovative Res. Sci. Eng. Technol. (An ISO 3297: 2007 Certified Organization) 3(4), 10906–10910 (2014)

    Google Scholar 

  17. Kong, W.W., Ranganath, S.: Towards subject independent continuous sign language recognition: a segment and merge approach. Pattern Recogn. 47, 1294–1308 (2014)

    Article  Google Scholar 

  18. Liwicki, S., Everingham, M.: Automatic recognition of finger spelled words in British sign language. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009. IEEE (2009)

    Google Scholar 

  19. MohanKumar, H.P., Nagendraswamy, H.S.: Change energy image for gait recognition: an approach based on symbolic representation. Int. J. Image Graphics Signal Proc. (IJIGSP) 6(4), 1–8 (2014)

    Article  Google Scholar 

  20. Murakami, K., Taguchi, H.: Gesture recognition using recurrent neural network. In: Actes de CHI 1991 Workshop on User Interface by Hand Gesture, pp 237–242. ACM (1991)

    Google Scholar 

  21. Nagendraswamy, H.S., Naresh, Y.G.: Representation and classification of medicinal plants: a symbolic approach based on fuzzy inference technique. In: Proceedings of the Second International Conference on Soft Computing for Problem Solving (SocProS 2012), 28–30 December 2012

    Google Scholar 

  22. Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)

    Article  MATH  Google Scholar 

  23. Oliva, A., Schyns, P.: Coarse blobs or fine edges? Evidence that information diagnosticity changes the perception of complex visual stimuli. Cogn. Psychol. 34, 72–107 (1997)

    Article  Google Scholar 

  24. Holden, E.-J., Lee, G., Owens, R.: Australian sign language recognition. Mach. Vis. Appl. 16(5), 312–320 (2005)

    Article  Google Scholar 

  25. Harling, P.A.: Gesture input using neural networks. Technical report, University of York, UK (1993)

    Google Scholar 

  26. Potter, M.C.: Meaning in visual search. Science 187(4180), 965–966 (1975)

    Article  Google Scholar 

  27. Rensink, R.A.: The dynamic representation of scenes. Vis. Cogn. 7, 17–42 (2000)

    Article  Google Scholar 

  28. Haralick, R.M., Shaipro, L.G.: Local invariant feature detectors: a survey. In: Computer and Robot Vision, vol. 2. Addison-Wesley Publishing Company, Boston (1993)

    Google Scholar 

  29. Ross, T.J.: Fuzzy Logic with Engineering Applications. Wiley, New York (2009)

    Google Scholar 

  30. Siagian, C., Itti, L.: Comparison of gist models in rapid scene categorization tasks. In: Proceedings of Vision Science Society Annual Meeting (VSS 2008), May 2008

    Google Scholar 

  31. Starner, T., Weaver, J., Pentland, A.: Real-time american sign language recognition using desk and wearable computer based video. IEEE Trans. Pattern Anal. Mach. Intell. 20(12), 1371–1375 (1998)

    Article  Google Scholar 

  32. Suraj, M.G., Guru, D.S.: Secondary diagonal FLD for fingerspelling Recognition. In: International Conference on Computing: Theory and Applications, International Conference on Computing: Theory and Applications, ICCTA 2007, pp. 693–697, (2007). doi:10.1109/ICCTA.2007

  33. Takahashi, T., Kishino, F.: Hand gesture coding based on experiments using a hand gesture interface device. SIGCHI Bull. 23, 67–74 (1991)

    Article  Google Scholar 

Download references

Acknowledgement

We would like to thank the students and the teaching staff of Sai Ranga Residential Boy’s School for Hearing Impaired, Mysore, and N K Ganpaiah Rotary School for physically challenged, Sakaleshpura, Hassan, Karnataka, INDIA, their immense support in the process of UoM-ISL Sign language dataset creation.

Author information

Authors and Affiliations

  1. Department of Studies in Computer Science, University of Mysore, Mysore, 570006, India

    H.S. Nagendraswamy, B.M. Chethana Kumara & R. Lekha Chinmayi

Authors
  1. H.S. Nagendraswamy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. B.M. Chethana Kumara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. Lekha Chinmayi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to B.M. Chethana Kumara .

Editor information

Editors and Affiliations

  1. Norwegian Univ. of Science & Technology, Trondheim, Norway

    Rajendra Prasath

  2. Intl Inst of Info Tech Hyderabad, Hyderabad, India

    Anil Kumar Vuppala

  3. V.H.N.S.N.College (Autonomous), Virudhunagar, Tamil Nadu, India

    T. Kathirvalavakumar

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Nagendraswamy, H., Chethana Kumara, B., Lekha Chinmayi, R. (2015). GIST Descriptors for Sign Language Recognition: An Approach Based on Symbolic Representation. In: Prasath, R., Vuppala, A., Kathirvalavakumar, T. (eds) Mining Intelligence and Knowledge Exploration. MIKE 2015. Lecture Notes in Computer Science(), vol 9468. Springer, Cham. https://doi.org/10.1007/978-3-319-26832-3_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-26832-3_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26831-6

  • Online ISBN: 978-3-319-26832-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation