Skip to main content
SpringerLink
Log in

Attention Based Segmentation and Recognition Algorithm for Hand Postures Against Complex Backgrounds

  • Chapter
  • First Online:
Computational Intelligence in Multi-Feature Visual Pattern Recognition

Part of the book series: Studies in Computational Intelligence ((SCI,volume 556))

Abstract

The Attention based Segmentation and Recognition (ASR) algorithm for hand postures against complex backgrounds is discussed in this chapter. The ASR algorithm can detect, segment and recognize multi-class hand postures. Visual attention, which is a cognitive process of selectively concentrating on a region of interest in visual field, helps humans to recognize objects in cluttered natural scenes. The ASR algorithm utilizes a Bayesian model of visual attention to generate a saliency map, and to detect and identify the hand region. Feature based visual attention is implemented using a combination of high level (shape, texture) and low level (color) image features. The shape and texture features are extracted from a skin similarity map, using a computational model of the ventral stream of visual cortex. The skin similarity map, which represents the similarity of each pixel to the human skin color in HSI color space, enhances the edges and shapes within the skin colored regions. The color features used are discretized chrominance components in HSI, YCbCr color spaces, and similarity-to-skin map. The hand postures are classified using shape and texture features, with a support vector machines classifier. The NUS hand posture dataset-II with 10 classes of complex background hand postures is utilized for testing the algorithm. The dataset contains hand postures from 40 subjects of different ethnicities. A total of 2,750 hand postures and 2,000 background images are available in the dataset. The hand postures vary in size and shape. The ASR algorithm is tested for hand detection and hand posture recognition using 10 fold cross-validation. The experimental results show that the algorithm has a person independent performance, and is reliable against variations in hand sizes and complex backgrounds.

Simple can be harder than complex: You have to work hard to get your thinking clean to make it simple. But it’s worth it in the end because once you get there, you can move mountains

Steve Jobs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Graph matching is considered as one of the most complex algorithms in vision based object recognition [2]. The complexity is due to the combinatorial nature of matching process.

  2. 2.

    The dataset is available for free download: http://www.vadakkepat.com/NUS-HandSet/.

  3. 3.

    V1, V2, V3, V4, and V5 are the visual areas in the visual cortex. V1 is the primary visual cortex. V2–V5 are the secondary visual areas, and are collectively termed as the extrastriate visual cortex.

  4. 4.

    Reference [23] for further explanation on \(S_1\) and \(C_1\) stages (layer 1 and 2).

  5. 5.

    The luminance color components are not utilized as these components are sensitive to skin color as well as lighting.

  6. 6.

    The dataset consists of hand postures by 40 subjects, with different ethnic origins.

  7. 7.

    400 images (1 image per class per subject) are considered. During the training phase the hand area is selected manually.

  8. 8.

    The dataset is available for academic research purposes: http://www.vadakkepat.com/NUS-HandSet/.

  9. 9.

    The dataset is available for free download: http://www.vadakkepat.com/NUS-HandSet/.

References

  1. V. Athitsos, S. Sclaroff, Estimating 3d hand pose from a cluttered image. IEEE Conf. Comput. Vis. Pattern Recogn. 2, 432–439 (2003)

    Google Scholar 

  2. E. Bienenstock, C. von der Malsburg, A neural network for invariant pattern recognition. Europhys. Lett. 4(1), 121–126 (1987)

    Article  Google Scholar 

  3. C. Bishop, Neural Networks for Pattern Recognition (Oxford, Oxford University Press, 1995)

    Google Scholar 

  4. J.M. Chaves-González, M.A. Vega-Rodrígueza, J.A. Gómez-Pulidoa, J.M. Sánchez-Péreza, Detecting skin in face recognition systems: a colour spaces study. Digit. Signal Process. 20(03), 806–823 (2010)

    Google Scholar 

  5. S. Chikkerur, T. Serre, C. Tan, T. Poggio, What and where: a bayesian inference theory of attention. Vis. Res. 50(22), 2233–2247 (2010)

    Google Scholar 

  6. P. Dayan, G.E. Hinton, R.M. Neal, The helmholtz machine. Neural Comput. 7(5), 889–904 (1995)

    Article  Google Scholar 

  7. L. Itti, C. Koch, Computational modelling of visual attention. Nat. Rev. Neurosci. 2(3), 194–203 (2001)

    Google Scholar 

  8. L. Itti, C. Koch, E. Niebur, A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Google Scholar 

  9. M.J. Jones, J.M. Rehg, Statistical color models with application to skin detection. IEEE Conf. Comput. Vis. Pattern Recogn. 1 (1999)

    Google Scholar 

  10. M. Kolsch, M. Turk, Robust hand detection. IEEE Conf. Autom. Face Gesture Recogn. 614–619 (2004)

    Google Scholar 

  11. K. Murphy, Bayes net toolbox for matlab (2003), http://code.google.com/p/bnt/

  12. E. Niebur, C. Koch, Computational architectures for attention, in The Attentive Brain, ed. by R. Parasuraman (Cambridge, MIT Press, 1998) pp. 163–186

    Google Scholar 

  13. E.J. Ong, R. Bowden, A boosted classifier tree for hand shape detection. IEEE Conf. Autom. Face Gesture Recogn. 889–894 (2004)

    Google Scholar 

  14. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Morgan Kaufmann Publishers, California, 1988)

    Google Scholar 

  15. S.L. Phung, A. Bouzerdoum, D. Chai, Skin segmentation using color pixel classification: analysis and comparison. IEEE Trans. Pattern Anal. Mach. Intell. 27(01), 148–154 (2005)

    Google Scholar 

  16. P.K. Pisharady, Computational intelligence techniques in visual pattern recognition. Ph.D. Thesis, National University of Singapore (2011)

    Google Scholar 

  17. P.K. Pisharady, P. Vadakkepat, A.P. Loh, Graph matching based hand posture recognition using neuro-biologically inspired features. International Conference on Control, Automation, Robotics and Vision (ICARCV) 2010 (Singapore), December 2010

    Google Scholar 

  18. P.K. Pisharady, P. Vadakkepat, A.P. Loh, Attention based detection and recognition of hand postures against complex backgrounds. Int. J. Comput. Vis. 101(03), 403–419 (2013)

    Google Scholar 

  19. P.K. Pisharady, P. Vadakkepat, A.P. Loh, Hand posture and face recognition using a fuzzy-rough approach. Int. J. Humanoid Rob. 07(03), 331–356 (2010)

    Google Scholar 

  20. T. Poggio, E. Bizzi, Generalization in vision and motor control. Nature 431, 768–774 (2004)

    Google Scholar 

  21. R. Rao, Bayesian inference and attentional modulation in the visual cortex. Neuro Rep. 16(16), 1843–1848 (2005)

    Google Scholar 

  22. M. Riesenhuber, T. Poggio, Hierarchical models of object recognition in cortex. Nat. Neurosci. 2(11), 1019–1025 (1999)

    Article  Google Scholar 

  23. T. Serre, L. Wolf, S. Bileschi, M. Riesenhuber, T. Poggio, Robust object recognition with cortex-like mechanisms. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 411–426 (2007)

    Article  Google Scholar 

  24. C. Siagian, L. Itti, Rapid biologically-inspired scene classification using features shared with visual attention. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 300–312 (2007)

    Google Scholar 

  25. J. Triesch, C. Malsburg, Sebastien marcel hand posture and gesture datasets: Jochen triesch static hand posture database (1996), http://www.idiap.ch/resource/gestures/

  26. J. Triesch, C. Malsburg, A system for person-independent hand posture recognition against complex backgrounds. IEEE Trans. Pattern Anal. Mach. Intell. 23(12), 1449–1453 (2001)

    Google Scholar 

  27. J. Triesch, C. Malsburg, Robust classification of hand postures against complex backgrounds. Proceedings of the Second International Conference on Automatic Face and Gesture Recognition (Killington, VT, USA), October 1996, pp. 170–175

    Google Scholar 

  28. J.K. Tsotsos, S.M. Culhane, Y.H. Wai, W.Y.K. Lai, N. Davis, F. Nuflo, Modelling visual attention via selective tuning. Artif. Intell. 78(1–2), 507–545 (1995)

    Google Scholar 

  29. Y. Wu, T.S. Huang, View-independent recognition of hand postures. IEEE Conf. Comput. Vis. Pattern Recogn. 2, 88–94 (2000)

    Google Scholar 

Download references

Acknowledgments

Figures and tables in this chapter are adapted from the following article with kind permission from Springer Science+Business Media: International Journal of Computer Vision, Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds, Vol.101, Issue No.3, 2013, Page Nos. 403-419, Pramod Kumar Pisharady, Prahlad Vadakkepat and Loh Ai Poh.

Author information

Authors and Affiliations

  1. Inst. of High Performance Computing, A*STAR, Singapore, Singapore

    Pramod Kumar Pisharady

  2. Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore

    Prahlad Vadakkepat & Loh Ai Poh

Authors
  1. Pramod Kumar Pisharady
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prahlad Vadakkepat
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Loh Ai Poh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pramod Kumar Pisharady .

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media Singapore

About this chapter

Cite this chapter

Pisharady, P.K., Vadakkepat, P., Poh, L.A. (2014). Attention Based Segmentation and Recognition Algorithm for Hand Postures Against Complex Backgrounds. In: Computational Intelligence in Multi-Feature Visual Pattern Recognition. Studies in Computational Intelligence, vol 556. Springer, Singapore. https://doi.org/10.1007/978-981-287-056-8_8

Download citation

  • DOI: https://doi.org/10.1007/978-981-287-056-8_8

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-287-055-1

  • Online ISBN: 978-981-287-056-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation