Encyclopedia of Computer Graphics and Games

Living Edition
| Editors: Newton Lee

Object Manipulation Using Real Hand Gesture for Augmented Reality Interior Design

  • Mohamad Yahya Fekri AladinEmail author
  • Ajune Wanis Ismail
  • Cik Suhaimi Yusof
  • Nur Syafiqah Safiee
Living reference work entry
DOI: https://doi.org/10.1007/978-3-319-08234-9_366-1



Augmented reality (AR) has the ability to blend the virtual and real objects. It enables users to interact with virtual objects or images with a real object as a medium of interaction. Hence, to provide natural and intuitive user interaction in AR environment, real hands are widely explored as an input. This entry explains the natural user interaction using real hand gesture in AR and explains the ARDeco application development process. The fundamental guide to developing AR application for interior design using real hand gestures with Leap Motion controller to interact with the UI and manipulate the objects with hand gestures, including color swatch and texture change panel for UIs and translation and rotation, scaling, create object, delete object, and clone object.


Augmented reality (AR) is a technology that allows computer-generated or digital information that includes...

This is a preview of subscription content, log in to check access.



This work was funded by UTM-GUP Funding Research Grants Scheme (Q.J130000.2528.19H89).


  1. Azuma, R.T.: A survey of augmented reality. Presence Teleop. Virt. 6(4), 355–385 (1997). Maxwell, J.C.: A Treatise on Electricity and Magnetism, 3rd ed., vol. 2. pp. 68–73. Clarendon, Oxford (1892)CrossRefGoogle Scholar
  2. Azuma, R.T.: Making augmented reality a reality. In: Applied Industrial Optics: Spectroscopy, Imaging and Metrology, pp. JTu1F-1. Optical Society of America (2017). https://bit.ly/2mOErYT
  3. Billinghurst, M.: Crossing the chasm. In: International Conference on Augmented Tele-Existence (ICAT 2001), pp. 5–7. ICAT, Tokyo (2001)Google Scholar
  4. Billinghurst, M., Clark, A., Lee, G.: A survey of augmented reality. Found. Trends® Human-Comput. Interact. 8(2–3), 73–272 (2015)CrossRefGoogle Scholar
  5. Buchmann, V., Violich, S., Billinghurst, M., Cockburn, A.: FingARtips: gesture-based direct manipulation in augmented reality. In: Proceedings of the 2nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia, pp. 212–221. ACM (2004)Google Scholar
  6. Evans, G., Miller, J., Pena, M. I., MacAllister, A., Winer, E.: Evaluating the Microsoft HoloLens through an augmented reality assembly application. In Degraded Environments: Sensing, Processing, and Display 2017 (Vol. 10197, p. 101970V). (2017). International Society for Optics and PhotonicsGoogle Scholar
  7. Hürst, W., Van Wezel, C.: Gesture-based interaction via finger tracking for mobile augmented reality. Multimed. Tools Appl. 62(1), 233–258 (2013)CrossRefGoogle Scholar
  8. Irawati, S., Green, S., Billinghurst, M., Duenser, A., Ko, H.: Move the couch where?: developing an augmented reality multimodal interface. In: Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 183–186. IEEE Computer Society (2006)Google Scholar
  9. Ismail, A.W., Sunar, M.S.: Multimodal fusion: gesture and speech input in augmented reality environment. In: Computational Intelligence in Information Systems, pp. 245–254. Springer, Cham (2015)CrossRefGoogle Scholar
  10. Ismail, A.W., Billinghurst, M., Sunar, M.S.: Vision-based technique and issues for multimodal interaction in augmented reality. In: Proceedings of the 8th International Symposium on Visual Information Communication and Interaction, pp. 75–82. ACM (2015, August)Google Scholar
  11. Kato, H., Billinghurst, M., Poupyrev, I., Imamoto, K., Tachibana, K.: Virtual object manipulation on a table-top AR environment. In: Augmented Reality, 2000. (ISAR 2000). Proceedings. IEEE and ACM International Symposium on, pp. 111–119. IEEE (2000)Google Scholar
  12. Lin, J., Schulze, J.P.: Towards naturally grabbing and moving objects in VR. Electron. Imaging. 2016(4), 1–6 (2016)CrossRefGoogle Scholar
  13. Magic Leap (2018). https://www.magicleap.com/
  14. Mann, S., Furness, T., Yuan, Y., Iorio, J., Wang, Z.: All Reality: Virtual, Augmented, Mixed (X), Mediated (X, Y), and Multimediated Reality (2018)Google Scholar
  15. Microsoft Hololens (2016). https://www.microsoft.com/en-us/hololens
  16. Nielsen, M., Störring, M., Moeslund, T.B., Granum, E.: A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In: International Gesture Workshop, pp. 409–420. Springer, Berlin, Heidelberg (2003, April)Google Scholar
  17. Piumsomboon, T., Altimira, D., Kim, H., Clark, A., Lee, G., Billinghurst, M.: Grasp-Shell vs gesture-speech: a comparison of direct and indirect natural inter-action techniques in augmented reality. In: Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium on, pp. 73–82. IEEE (2014, September)Google Scholar
  18. Syafiqa, N.S., Ismail, A.W.: AR Home Deco: Virtual Object Manipulation Technique Using Hand Gesture in Augmented Reality, UTM Computing Proceeding, vol 3, pp. 1–6 (2018). https://bit.ly/2nODkrY
  19. Szalavári, Z., Schmalstieg, D., Fuhrmann, A., Gervautz, M.: “Studierstube”: an environment for collaboration in augmented reality. Virtual Reality. 3(1), 37–48 (1998)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Mohamad Yahya Fekri Aladin
    • 1
    • 2
    Email author
  • Ajune Wanis Ismail
    • 1
    • 2
  • Cik Suhaimi Yusof
    • 1
    • 2
  • Nur Syafiqah Safiee
    • 1
    • 2
  1. 1.Mixed and Virtual Reality Research Lab, VicubelabUniversiti Teknologi MalaysiaJohor BahruMalaysia
  2. 2.School of Computing, Faculty of EngineeringUniversiti Teknologi MalaysiaJohor BahruMalaysia