Abstract
This paper presents a novel method for markerless hand gesture recognition with a recently developed depth sensor. The proposed method encompasses a collection of techniques that enable the modeling and recognition of hand gestures. Hand detection and location are processed with the depth information acquired from a depth sensor. Then, the hand is robustly segmented in cluttered background without any marker around using only depth information. A convex shape decomposition method based on Radius Morse function is proposed for hand shape decomposition in real time. Hand palm and fingertips are recognized based on the hand shape decomposition and hand features. A prototype implementation of the developed system operates on 640x480 live video with both depth image and color image in real time on a conventional processor. Representative experimental results prove the accuracy, efficiency and robustness of our method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Wachs, J., Kölsch, M., Stern, H., Edan, Y.: Vision-based hand-gesture applications. Communications of the ACM 54(2), 60–71 (2011)
Wang, R., Popović, J.: Real-time hand-tracking with a color glove. ACM Transactions on Graphics (TOG) 28(3) (2009)
Lee, T., Hollerer, T.: Handy AR: Markerless inspection of augmented reality objects using fingertip tracking. In: 11th IEEE International Symposium on Wearable Computers, pp. 83–90. IEEE (2007)
Lee, T., Hollerer, T.: Multithreaded hybrid feature tracking for markerless augmented reality. IEEE Transactions on Visualization and Computer Graphics 15(3), 355–368 (2009)
Bretzner, L., Laptev, I., Lindeberg, T.: Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering. In: Fifth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 423–428. IEEE (2002)
Argyros, A.A., Lourakis, M.I.A.: Real-Time Tracking of Multiple Skin-Colored Objects with a Possibly Moving Camera. In: Pajdla, T., Matas, J. (eds.) ECCV 2004, Part III. LNCS, vol. 3023, pp. 368–379. Springer, Heidelberg (2004)
Argyros, A.A., Lourakis, M.I.A.: Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse. In: Huang, T.S., Sebe, N., Lew, M., Pavlović, V., Kölsch, M., Galata, A., Kisačanin, B. (eds.) HCI/ECCV 2006. LNCS, vol. 3979, pp. 40–51. Springer, Heidelberg (2006)
Ren, Z., Yuan, J., Zhang, Z.: Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 1093–1096. ACM (2011)
Lindeberg, T.: Feature detection with automatic scale selection. International Journal of Computer Vision 30(2), 79–116 (1998)
Fang, Y., Cheng, J., Wang, K., Lu, H.: Hand gesture recognition using fast multi-scale analysis. In: Fourth International Conference on Image and Graphics, ICIG 2007, pp. 694–698. IEEE (2007)
Fang, Y., Wang, K., Cheng, J., Lu, H.: A real-time hand gesture recognition method. In: IEEE International Conference on Multimedia and Expo, pp. 995–998. IEEE (2007)
Liu, X., Fujimura, K.: Hand gesture recognition using depth data. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 529–534. IEEE (2004)
Oikonomidis, I., Kyriazis, N., Argyros, A.: Efficient model-based 3D tracking of hand articulations using kinect. In: Procs. of BMVC, Dundee, UK, August 29-September 10 (2011) [547]
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, vol. 2, pp. 1297–1304 (2011)
Cai, Q., Gallup, D., Zhang, C., Zhang, Z.: 3D Deformable Face Tracking with a Commodity Depth Camera. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 229–242. Springer, Heidelberg (2010)
Hackenberg, G., McCall, R., Broll, W.: Lightweight palm and finger tracking for real-time 3D gesture control. In: IEEE Virtual Reality Conference, VR, pp. 19–26. IEEE (2011)
Schwarz, L., Mkhitaryan, A., Mateus, D., Navab, N.: Estimating human 3D pose from time-of-flight images based on geodesic distances and optical flow. In: IEEE International Conference on Automatic Face & Gesture Recognition and Workshops, FG 2011, pp. 700–706. IEEE (2011)
Zhu, J., Wang, L., Yang, R., Davis, J.: Fusion of time-of-flight depth and stereo for high accuracy depth maps. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)
Daribo, I., Saito, H.: A novel inpainting-based layered depth video for 3DTV. IEEE Transactions on Broadcasting (99), 533–541 (2011)
Liu, H., Liu, W., Latecki, L.: Convex shape decomposition. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 97–104. IEEE (2010)
Telea, A.: An image inpainting technique based on the fast marching method. Journal of Graphics Tools 9(1), 23–34 (2004)
Kopf, J., Cohen, M., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. ACM Transactions on Graphics 26(3) (2007)
Lien, J., Amato, N.: Approximate convex decomposition of polygons. Computational Geometry 35(1), 100–123 (2006)
Mi, X., DeCarlo, D.: Separating parts from 2D shapes using relatability. In: IEEE 11th International Conference on Computer Vision, pp. 1–8. IEEE (2007)
Cole-McLaughlin, K., Edelsbrunner, H., Harer, J., Natarajan, V., Pascucci, V.: Loops in reeb graphs of 2-manifolds. In: Proceedings of the Nineteenth Annual Symposium on Computational Geometry, pp. 344–350. ACM (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Qin, S., Zhu, X., Yu, H., Ge, S., Yang, Y., Jiang, Y. (2012). Real-Time Markerless Hand Gesture Recognition with Depth Camera. In: Lin, W., et al. Advances in Multimedia Information Processing – PCM 2012. PCM 2012. Lecture Notes in Computer Science, vol 7674. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34778-8_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-34778-8_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34777-1
Online ISBN: 978-3-642-34778-8
eBook Packages: Computer ScienceComputer Science (R0)