Interaction on Augmented Reality with Finger Detection and Hand Movement Recognition

  • Mohammad Fadly SyahputraEmail author
  • Siti FatimahEmail author
  • Romi Fadillah RahmatEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10851)


Indonesia’s rare animals are animals whose habitats exist only in Indonesia and are endangered by IUCN (International Union for Conservation of Nature and Natural Resources). In this study, we applied Augmented Reality (AR) technology to represent the endangered species in Indonesia combined with image processing techniques as the interaction between users and the virtual objects of endangered animals. The initial stage is the separation of the desired object from the background with the help of HSV colour method, as well as detecting contour with contour detector. As for the calculation of the number of fingers we are using convex hull and convexity defects. The results of this stage is the number of fingers that will be used as a reference selection of endangered animal. These series of processes generates an augmented reality application where users can view and provide instructions with virtual objects of endangered species of Indonesia. This can provide a different experience for users to learn about Indonesia’s endangered species.


Rare animal in Indonesia Augmented reality Image processing Finger detection 


  1. 1.
    Iqbal, M., Kurnia, M.P., Susanti, E.: Tinjauan Yuridis Terhadap Kepemilikan dan Penjualan Satwa Langka Izin di Indonesia. (Analysis of Endangerd Animals and its Selling in Indonesia) Skripsi. Universitas Mulawarman (2014)Google Scholar
  2. 2.
    Brotos, A.: Interactive Augmented Reality Panel Interface for Android. Stanford University (2015)Google Scholar
  3. 3.
    Amin, D., Govilkar, S.: Comprative study of augmented reality SDK’s. Int. J. Comput. Sci. Appl. (IJCSA) 5(1), 11–26 (2015)Google Scholar
  4. 4.
    Benabderrahim, K., Bouhlel, M,S.: Detecting and tracking the hand to create an augmented reality system. IOSR J. Comput. Eng. (IOSR-JCE) (2014)Google Scholar
  5. 5.
    Bhatt, R., Fernandes, N., Dhage, A.: Vision based hand gesture recognition for human computer interaction. Int. J. Eng. Sci. Innov. Technol. (IJESIT) 2(3), 110–115 (2013)Google Scholar
  6. 6.
    Bikos, Mario, Itoh, Yuta, Klinker, Gudrun, Monstakas, Konstantinos: An Interactive Augmented Reality Chess Game using Bare-H and Pinch Gestures. University of Patras, Tesis (2014)Google Scholar
  7. 7.
    Dhawan, A., Honrao, V.: Implementation of hand detection based techniques for human computer interaction. Int. J. Comput. Appl. (0975 – 8887) 72(17), 6–13 (2013)Google Scholar
  8. 8.
    Seo, D.W., Lee, J.Y.: Direct Hand Touchable Interactions in Augmented Reality Environments for Natural and Intuitive User Experiences. Chonnam National University, Tesis (2013)Google Scholar
  9. 9.
    Ramjan, M.R., Sandip, R.M., Uttam, P.S., Srimani, W.S.: Dynamic hand gesture recognition and detection for real time using human computer interaction. Int. J. Adv. Res. Comput. Sci. Manag. Stud. (IJARCSMS) 2(3), 425–430 (2014)Google Scholar
  10. 10.
    Syahputra, Mohammad Fadly, Siregar, Ridho K., Rahmat, Romi Fadillah: Finger recognition as interaction media in augmented reality for historical buildings in matsum and kesawan regions of Medan city. In: De Paolis, Lucio Tommaso, Bourdot, Patrick, Mongelli, Antonio (eds.) AVR 2017. LNCS, vol. 10325, pp. 243–250. Springer, Cham (2017). Scholar
  11. 11.
    Lee, T., Tobias, H.: Handy AR: Markerless Inspection of Augmented Reality Object Using FingerTip Tracking . Thesis. University of California (2007)Google Scholar
  12. 12.
    Lee, H., Tateyama, Y., Ogi, T.: Hand gesture recognition using blob detection for immersive projection display system. World Academy of Science, Engineering and Technology. Int. J. Computer, Electr. Autom. Control Inf. Eng. 6(2), 260–263 (2012)Google Scholar
  13. 13.
    Yesugade, K.D., Salunke, S., Shinde, K., Gaikwad, S., Shingare, M.: Hand motion recognition. Int. J. Technol. Exploring Eng. (IJITEE) 3(11), 55–61 (2014)Google Scholar
  14. 14.
    Youssef, M.M., Asari, K.V., Tompkins, R.C., Foytik, J.: Hull convexity defects features for human activity recognition. IEEE Applied Imagery Pattern Recognition Workshop (AIPR), pp. 1–7 (2010)Google Scholar
  15. 15.
    Rahmat, R.F., Chairunnisa, T., Gunawan, D., Sitompul, O.S.: Skin color segmentation using multi-color space threshold. In: 3rd International Conference on Computer and Information Sciences (ICCOINS), Kuala Lumpur, pp. 391–396 (2016)Google Scholar
  16. 16.
    Rai, M., Bhootna, V., Yadav, R.K.: Performance based algorithm for the detection and extraction of human skin. First International Conference on Futuristic Trend in Computational Analysis and Knowledge Management (ABLAZE), pp. 127–131 (2015)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Information Technology, Faculty of Computer Science and Information TechnologyUniversitas Sumatera UtaraMedanIndonesia

Personalised recommendations