Skip to main content
SpringerLink
Log in

Curve Matching from the View of Manifold for Sign Language Recognition

  • Conference paper
  • First Online:
Computer Vision - ACCV 2014 Workshops (ACCV 2014)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9010))

Included in the following conference series:

Abstract

Sign language recognition is a challenging task due to the complex action variations and the large vocabulary set. Generally, sign language conveys meaning through multichannel information like trajectory, hand posture and facial expression simultaneously. Obviously, trajectories of sign words play an important role for sign language recognition. Although the multichannel features are helpful for sign representation, this paper only focuses on the trajectory aspect. A method of curve matching based on manifold analysis is proposed to recognize isolated sign language word with 3D trajectory captured by Kinect. From the view of manifold, the main structure of the curve is found by the intrinsic linear segments, which are characterized by some geometric features. Then the matching between curves is transformed into the matching between two sets of sequential linear segments. The performance of the proposed curve matching strategy is evaluated on two different sign language datasets. Our method achieves a top-1 recognition rate of 78.3 % and 61.4 % in a 370 daily words dataset and a large dataset containing 1000 vocabularies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Murakami, K., Taguchi, H.: Gesture recognition using recurrent neural networks. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 1991, pp. 237–242. ACM, New York (1991)

    Google Scholar 

  2. Huang, C.L., Huang, W.Y., Lien, C.C.: Sign language recognition using 3-d hopfield neural network. In: Proceedings of the International Conference on Image Processing, vol. 2, pp. 611–614 (1995)

    Google Scholar 

  3. Kim, J.S., Jang, W., Bien, Z.: A dynamic gesture recognition system for the korean sign language (ksl). IEEE Trans. Syst. Man Cybern. Part B: Cybern. 26, 354–359 (1996)

    Article  Google Scholar 

  4. Grobel, K., Assan, M.: Isolated sign language recognition using hidden markov models. In: 1997 IEEE International Conference on Systems, Man, and Cybernetics, Computational Cybernetics and Simulation, vol. 1, pp. 162–167 (1997)

    Google Scholar 

  5. Starner, T., Weaver, J., Pentland, A.: Real-time american sign language recognition using desk and wearable computer based video. IEEE Trans. Pattern Anal. Mach. Intell. 20, 1371–1375 (1998)

    Article  Google Scholar 

  6. Mokhtarian, F., Mackworth, A.: Scale-based description and recognition of planar curves and two-dimensional shapes. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–8, 34–43 (1986)

    Article  Google Scholar 

  7. Zuliani, M., Bhagavathy, S., Manjunath, B., Kenney, C.: Affine-invariant curve matching. In: 2004 International Conference on Image Processing, ICIP 2004, vol. 5, pp. 3041–3044 (2004)

    Google Scholar 

  8. Efrat, A., Fan, Q., Venkatasubramanian, S.: Curve matching, time warping, and light fields: New algorithms for computing similarity between curves. J. Math. Imaging Vis. 27, 203–216 (2007)

    Article  MathSciNet  Google Scholar 

  9. Pajdla, T., Gool, L.V.: Matching of 3-d curves using semi-differential invariants. In: Proceedings of the Fifth International Conference on Computer Vision, pp. 390–395 (1995)

    Google Scholar 

  10. Kishon, E., Hastie, T., Wolfson, H.: 3-d curve matching using splines. In: Faugeras, O. (ed.) ECCV 1990. LNCS, vol. 427, pp. 589–591. Springer, Heidelberg (1990)

    Chapter  Google Scholar 

  11. Shahraray, B., Anderson, D.: Uniform resampling of digitized contours. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–7, 674–681 (1985)

    Article  Google Scholar 

  12. Wobbrock, J.O., Wilson, A.D., Li, Y.: Gestures without libraries, toolkits or training: A \({\$}\)1 recognizer for user interface prototypes. In: Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology, UIST 2007, pp. 159–168. ACM, New York (2007)

    Google Scholar 

  13. Wang, R., Shan, S., Chen, X., Chen, J., Gao, W.: Maximal linear embedding for dimensionality reduction. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1776–1792 (2011)

    Article  Google Scholar 

  14. Bahlmann, C., Burkhardt, H.: The writer independent online handwriting recognition system frog on hand and cluster generative statistical dynamic time warping. IEEE Trans. Pattern Anal. Mach. Intell. 26, 299–310 (2004)

    Article  Google Scholar 

  15. Martens, R., Claesen, L.: On-line signature verification by dynamic time-warping. In: Proceedings of the 13th International Conference on Pattern Recognition, vol. 3, pp. 38–42 (1996)

    Google Scholar 

  16. Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust. Speech Signal Process. 26, 43–49 (1978)

    Article  MATH  Google Scholar 

  17. Ren, Z., Meng, J., Yuan, J., Zhang, Z.: Robust hand gesture recognition with kinect sensor. In: Proceedings of the 19th ACM International Conference on Multimedia, MM 2011, pp. 759–760. ACM, New York (2011)

    Google Scholar 

  18. Tong, J., Zhou, J., Liu, L., Pan, Z., Yan, H.: Scanning 3d full human bodies using kinects. IEEE Trans. Visual Comput. Graphics 18, 643–650 (2012)

    Article  Google Scholar 

  19. Zafrulla, Z., Brashear, H., Starner, T., Hamilton, H., Presti, P.: American sign language recognition with the kinect. In: Proceedings of the 13th International Conference on Multimodal Interfaces, ICMI 2011, pp. 279–286, ACM, New York (2011)

    Google Scholar 

  20. Sun, C., Zhang, T., Bao, B.K., Xu, C., Mei, T.: Discriminative exemplar coding for sign language recognition with kinect. IEEE Trans. Cybern. 43, 1418–1428 (2013)

    Article  Google Scholar 

  21. Al-Hajj Mohamad, R., Likforman-Sulem, L., Mokbel, C.: Combining slanted-frame classifiers for improved hmm-based arabic handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1165–1177 (2009)

    Article  Google Scholar 

  22. Chai, X., Li, G., Lin, Y., Xu, Z., Tang, Y., Chen, X., Zhou, M.: Sign language recognition and translation with kinect. In: IEEE Conference on AFGR (2013)

    Google Scholar 

  23. Wang, J., Liu, Z., Wu, Y., Yuan, J.: Mining actionlet ensemble for action recognition with depth cameras. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1290–1297 (2012)

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by the Microsoft Research Asia, and Natural Science Foundation of China under contract Nos. 61303170, 61472398, and the FiDiPro program of Tekes.

Author information

Authors and Affiliations

  1. Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing, 100190, China

    Yushun Lin, Xiujuan Chai & Xilin Chen

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Yushun Lin

  3. Institute of Information Engineering, CAS, Beijing, 100093, China

    Yu Zhou

Authors
  1. Yushun Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiujuan Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xilin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yushun Lin .

Editor information

Editors and Affiliations

  1. Center for Visual Information Technology, International Institute of Information Technology, Hyderabad, India

    C. V. Jawahar

  2. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Shiguang Shan

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Lin, Y., Chai, X., Zhou, Y., Chen, X. (2015). Curve Matching from the View of Manifold for Sign Language Recognition. In: Jawahar, C., Shan, S. (eds) Computer Vision - ACCV 2014 Workshops. ACCV 2014. Lecture Notes in Computer Science(), vol 9010. Springer, Cham. https://doi.org/10.1007/978-3-319-16634-6_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-16634-6_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-16633-9

  • Online ISBN: 978-3-319-16634-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation