Encyclopedia of Computer Graphics and Games

Living Edition
| Editors: Newton Lee

Interactive Augmented Reality Pop-Up Book with Natural Gesture Interaction for Handheld

  • Muhammad Nur Affendy Nor’aEmail author
  • Ajune Wanis Ismail
  • Mohamad Yahya Fekri Aladin
Living reference work entry
DOI: https://doi.org/10.1007/978-3-319-08234-9_365-1



Handheld augmented reality (AR) has been widely used with smart and portable device in the applications such as education, games, visual experience, and information visualization. However, most of the handheld applications do not fully support natural interaction, and the existing 3D pop-up book has used touch-based to interact with 3D content. Therefore, this entry describes a fundamental to design an interactive AR pop-up book with natural gesture interaction using real hand. Subsequently, the real hand gesture tracking in handheld AR is explored to examine how it can track user’s hands in real time. Thus, this entry describes about gesture interaction to allow the user to directly interact with the virtual objects. The user feels more realistic to interact with 3D objects using their bare hands on 3D pop-up book.


Augmented reality (AR)33 is a technology that...

This is a preview of subscription content, log in to check access.


  1. Azuma, R., Behringer, R., Feiner, S., Julier, S., Macintyre, B.: Recent advances. In EEE Computer Graphics and Applications, 2011(December), 1–27 (2001)Google Scholar
  2. Bai, H., Gao, L., El-Sana, J., Billinghurst, M.: Markerless 3D gesture-based interaction for handheld augmented reality interfaces. In Mixed and Augmented Reality (ISMAR), 2013 IEEE International Symposium on, pp. 1–6. IEEE (2013)Google Scholar
  3. Billinghurst, M., Kato, H., Poupyrev, I. Tangible Augmented Reality. ACM SIGGRAPH ASIA 2008 Courses, 7, pp. 1–10 (2008)Google Scholar
  4. Billinghurst, M., Kato, H., Poupyrev, I.: The MagicBook: a transitional AR interface. Comput. Graph. 25(5), 745–753 (2001)CrossRefGoogle Scholar
  5. Chun, J., Lee, S.: A vision-based 3D hand interaction for marker-based AR. Int J Multimed Ubiquit Eng. 7(3), 51–58 (2012)Google Scholar
  6. Clark, A., Dünser, A., Grasset, R.: An interactive augmented reality coloring book. In: Mixed and Augmented Reality (ISMAR), 2011 10th IEEE International Symposium on, pp. 259–260. IEEE (2011)Google Scholar
  7. Cohen, P.R., Dalrymple, M., Moran, D.B., Pereira, F.C., Sullivan, J.W., Cohen, P.R., Sullivan, J.W.: Synergistic use of direct manipulation and natural language. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Wings for the Mind – CHI ’89, vol. 20, pp. 227–233. ACM Press, New York (1989)CrossRefGoogle Scholar
  8. Guna, J., Jakus, G., Pogačnik, M., Tomažič, S., Sodnik, J.: An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors. 14(2), 3702–3720 (2014)CrossRefGoogle Scholar
  9. Hilliges, O., Kim D., Izadi S., Molyneaux D., Hodges S.E., Butler D.A.: Augmented reality with direct user interaction. U.S. Patent 9,891,704, issued February 13 (2018)Google Scholar
  10. Ismail, A.W., Sunar, M.S.: Intuitiveness 3D objects interaction in augmented reality using S-PI algorithm. Indones J Electr Eng Comput Sci. 11(7), 3561–3567 (2013)Google Scholar
  11. Ismail, A.W., Sunar, M.S.: Multimodal fusion: gesture and speech input in augmented reality environment. In: Computational Intelligence in Information Systems: Proceedings of the Fourth INNS Symposia Series on Computational Intelligence in Information Systems (INNS-CIIS 2014), vol. 331, p. 245. Springer, Cham (2014)CrossRefGoogle Scholar
  12. Kim, M., Lee, J.Y.: Touch and hand gesture-based interactions for directly manipulating 3D virtual objects in mobile augmented reality. Multimed. Tools Appl. 75, 16529 (2016)CrossRefGoogle Scholar
  13. Lv, Z., Halawani, A., Feng, S., Ur Réhman, S., Li, H.: Touch-less interactive augmented reality game on vision-based wearable device. Pers. Ubiquit. Comput. 19(3–4), 551–567 (2015)CrossRefGoogle Scholar
  14. Markouzis, D., & Fessakis, G.: Interactive storytelling and mobile augmented reality applications for learning and entertainment – a rapid prototyping perspective. In: Interactive Mobile Communication Technologies and Learning (IMCL), 2015 International Conference on, pp. 4–8. IEEE (2015)Google Scholar
  15. Network, P.U.: How to Create an Online Multiplayer Game with Photon Unity Networking (2015)Google Scholar
  16. Piumsomboon, T., Altimira, D., Kim, H., Clark, A., Lee, G., Billinghurst, M.: Grasp-Shell vs gesture-speech: a comparison of direct and indirect natural interaction techniques in augmented reality. In ISMAR 2014 – IEEE International Symposium on Mixed and Augmented Reality – Science and Technology 2014, Proceedings, pp. 73–82 (2014)Google Scholar
  17. Samini, A., Palmerius, K.L.: A study on improving close and distant device movement pose manipulation for hand-held augmented reality. In The 22nd ACM Symposium on Virtual Reality Software and Technology (VRST), Munich, Germany, November 02-04, 2016 (pp. 121–128). ACM Press (2016)Google Scholar
  18. Vuibert, V., Stuerzlinger, W., Cooperstock, J.R.: Evaluation of docking task performance using mid-air interaction techniques. In: Proceedings of the 3rd ACM Symposium on Spatial User Interaction (pp. 44–52). ACM (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Muhammad Nur Affendy Nor’a
    • 1
    • 2
    Email author
  • Ajune Wanis Ismail
    • 1
    • 2
  • Mohamad Yahya Fekri Aladin
    • 1
    • 2
  1. 1.Mixed and Virtual Reality Research Lab, VicubelabUniversiti Teknologi MalaysiaJohor BahruMalaysia
  2. 2.School of Computing, Faculty of EngineeringUniversiti Teknologi MalaysiaJohor BahruMalaysia