A New Approach for Hand Augmentation Based on Patch Modelling

  • Omer Rashid Ahmad
  • Ayoub Al-Hamadi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8192)


In this paper, a novel patch-based modelling approach is proposed to augment the 3D models over the non-planar structure of hand. We present a robust real-time framework which first segment the hand using skin color information. To offer the subjects flexible interface (i.e., no restriction of covered arm), segmented skin-region is refined through distance descriptor which gives the hand’s palm. Next, we measure the geometrical components from the hand’s structure and compute the hand’s curvature to detect the fingertips. The palm centroid and detected fingertips contribute to the patch-based modelling for which optimal path is derived for every detected fingertip. Further, we compute the patches using the path points for every two neighbor detected fingers and then the camera poses are estimated for these patches. These different camera poses are finally combined to generate a single camera pose, and then 3D models are augmented on non-planar hand geometry. The experimental results show that our proposed approach is capable to detect hand, palm centroid and fingertips, as well as integrate the generated patches for augmented reality application in real situations which proves its applicability and usability in the domain of Human Computer Interaction.


Augmented Reality Hand and Fingertip Detection Pose Estimation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Azuma, R.T.: A survey of augmented reality. Presence: Teleoperators and Virtual Environments 6(4), 355–385 (1997)CrossRefGoogle Scholar
  2. 2.
    Borgefors, G.: Distance transformations in digital images. Computer Vision, Graphics, and Image Processing 34(3), 344–371 (1986)CrossRefGoogle Scholar
  3. 3.
    Buchmann, V., Violich, S., Billinghurst, M., Cockburn, A.: Fingartips: Gesture based direct manipulation in augmented reality. In: 2nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia (GRAPHITE 2004), pp. 212–221 (2004)Google Scholar
  4. 4.
    Chu, S., Tanaka, J.: Hand gesture for taking self portrait. In: Jacko, J.A. (ed.) Human-Computer Interaction, Part II, HCII 2011. LNCS, vol. 6762, pp. 238–247. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Chun, J.C., Lee, B.S.: Dynamic manipulation of a virtual object in marker-less ar system based on both human hands. Transactions on Internet and Information Systems 4(4), 618–632 (2010)Google Scholar
  6. 6.
    Erol, A., Bebis, G., Nicolescu, M., Boyle, R.D., Twombly, X.: Vision-based hand pose estimation: A review. Computer Vision and Image Understanding 108(1-2), 52–73 (2007)CrossRefGoogle Scholar
  7. 7.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, New York (2003)zbMATHGoogle Scholar
  8. 8.
    Lee, B.S., Chun, J.C.: Interactive manipulation of augmented objects in marker-less ar using vision-based hand mouse. In: International Conference on Information Technology (ITNG), pp. 398–403 (2010)Google Scholar
  9. 9.
    Lee, T., Hoellerer, T.: Handy ar: Markerless inspection of augmented reality objects using fingertip tracking. In: ISWC 2007, pp. 83–90 (2007)Google Scholar
  10. 10.
    Lee, T., Hoellerer, T.: Hybrid feature tracking and user interaction for markerless augmented reality. In: IEEE Virtual Reality, pp. 145–152 (2008)Google Scholar
  11. 11.
    Malassiotis, S., Strintzis, M.: Real-time hand posture recognition using range data. Image and Vision Computing 26(7), 1027–1037 (2008)CrossRefGoogle Scholar
  12. 12.
    Ng, K.P., Tan, G.Y., Wong, Y.P.: Vision-based hand detection for registration of virtual objects in augmented reality. International Journal of Future Computer and Communication 2(5), 423–427 (2013)Google Scholar
  13. 13.
    Noell, T., Pagani, A., Stricker, D.: Markerless camera pose estimation - an overview. In: Visualization of Large and Unstructured Data Sets - Applications in Geospatial Planning, Modeling and Engineering. 4, vol. 19, pp. 45–54 (2011)Google Scholar
  14. 14.
    Radkowski, R., Stritzke, C.: Interactive hand gesture-based assembly for augmented reality applications. In: Proceedings of the 2012 International Conference on Advances in Computer-Human Interactions, pp. 303–308. IEEE (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Omer Rashid Ahmad
    • 1
  • Ayoub Al-Hamadi
    • 1
  1. 1.Institute for Information Technology and CommunicationsOtto-von-Guericke University MagdeburgGermany

Personalised recommendations