Skip to main content
SpringerLink
Log in

Interaction techniques in desktop virtual environment: the study of visual feedback and precise manipulation method

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Gesture-based systems allow users to interact with a virtual reality application in a natural way. Visual feedback for the gesture-based interaction technique has an impact on the performance and the hand instability making the manipulation of the object less precise. This paper investigated two new interaction techniques in a virtual environment. It describes the influence of natural and non-natural virtual feedback in the selection process using the GITDVR-G interaction technique, which consists of a grasping visual feedback. The GITDVR-G was evaluated in a virtual knee surgery training system. The results showed that it was effective in terms of the task completion time, and that the participants preferred the natural grasping visual feedback. Besides that, the precise manipulation in a newly-designed interaction technique (Precise GITDVR-G) was evaluated. The Precise GITDVR-G includes a normal manipulation mode and a precise manipulation mode that can be triggered by hand gestures. During the precise manipulation mode, an inset view will appear and move with the selected object to provide a better view to users, while the movements of the virtual hand are scaled down to improve the precision. Four different configurations of the precise manipulation technique were evaluated, and the results showed that the unimanual control method with an inset view performed better in terms of the task performance time and the subjective feedback. The finding suggested that the realistic virtual grasping visual feedback can be applied in a virtual hand interaction technique, and that the inset view feature is helpful in the precise manipulation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Argelaguet F, Andujar C (2009) Visual feedback techniques for virtual pointing on stereoscopic displays. In: Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology, Kyoto, Japan, 2009. ACM, 1643966, pp 163-170. https://doi.org/10.1145/1643928.1643966

  2. Ascension Technology Corporation (2002) Flock of birds: installation and operation guide. Burlington. http://grva.lamce.coppe.ufrj.br/hardware/hw_drivers/flockofbirds/Flock_of_Birds_Manual-RevB.pdf. Accessed 12 Jun 2011

  3. Bowman DA (1996) Conceptual Design Space: Beyond Walk-through to Immersive Design. In: Designing Digital Space: An Architect's Guide to Virtual Reality. John Wiley & Sons, New York, pp 225–236

    Google Scholar 

  4. Bowman DA, Hodges LF (1997) An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In: Proceedings of the 1997 symposium on Interactive 3D graphics, Providence, Rhode Island, 1997. ACM, 253301, pp 35-38. https://doi.org/10.1145/253284.253301

  5. Bowman DA, Kruijff E, LaViola JJ, Poupyrev I (2001) An Introduction to 3-D User Interface Design. Presence: Teleoper Virtual Environ 10(1):96–108. https://doi.org/10.1162/105474601750182342

    Article  Google Scholar 

  6. Bowman DA, Kruijff E, LaViola JJ, Poupyrev I (2004) 3D User Interfaces: Theory and Practice. Addison Wesley Longman Publishing. http://www.amazon.com/3D-User-Interfaces-Practice-paperback/dp/0321980042

  7. Brooks FP, Ouh-Young M, Batter JJ, Kilpatrick PJ (1990) Project GROPEHaptic displays for scientific visualization. In: Proceedings of the 17th annual conference on Computer graphics and interactive techniques, Dallas, TX, USA, 1990. ACM, 97899, pp 177-185. https://doi.org/10.1145/97879.97899

  8. Burdea G, Coiffet P (1994) Virtual reality technology. John Wiley & Son, New York

  9. Fabiani L, Burdea G, Langrana N, Gomez D (1996) Human interface using the Rutgers Master II force feedback interface. In: Proceedings of the IEEE Virtual Reality Annual International Symposium, 1996, 30 Mar-3 Apr 1996. pp 54-59. https://doi.org/10.1109/VRAIS.1996.490510

  10. Faieza AAP, Pham DT, Sulaiman S, Napsiah I, Ariffin MK, BTHT B (2008) Visual Feedback and Pseudo-Haptic Feedback Improve Manual Lifting Performance. Jurnal Teknologi 49(D):157–171

    Google Scholar 

  11. Fifth Dimension Technologies (2004) 5DT data glove ultra series: user’s manual. India. http://5dt.com/downloads/dataglove/ultra/5DT%20Data%20Glove%20Ultra%20-%20Manual.pdf. Accessed 8 April 2011

  12. Forsberg A, Herndon K, Zeleznik R (1996) Aperture based selection for immersive virtual environments. In: Proceedings of the 9th annual ACM symposium on User interface software and technology, Seattle, Washington, United States, 1996. ACM, 237105, pp 95-96. https://doi.org/10.1145/237091.237105

  13. Frees S, Kessler GD, Kay E (2007) PRISM interaction for enhancing control in immersive virtual environments. ACM Trans Comput-Hum Interact 14(1):2. https://doi.org/10.1145/1229855.1229857

    Article  Google Scholar 

  14. Grossman T, Balakrishnan R (2005) The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor's activation area. In: Proceedings of the SIGCHI conference on Human factors in computing systems, Portland, Oregon, USA, 2005. ACM, 1055012, pp 281-290. https://doi.org/10.1145/1054972.1055012

  15. Grossman T, Balakrishnan R (2006) The design and evaluation of selection techniques for 3D volumetric displays. In: Proceedings of the 19th annual ACM symposium on User interface software and technology, Montreux, Switzerland, 2006. ACM, 1166257, pp 3-12. https://doi.org/10.1145/1166253.1166257

  16. Hilliges O, Izadi S, Wilson AD, Hodges S, Garcia-Mendoza A, Butz A (2009) Interactions in the air: adding further depth to interactive tabletops. In: Proceedings of the 22nd annual ACM symposium on User interface software and technology, Victoria, BC, Canada, 2009. ACM, 1622203, pp 139-148. https://doi.org/10.1145/1622176.1622203

  17. Hou W (2008) The Three-Dimensional User Interface. In: Pinder S (ed) Advances in Human Computer Interaction. InTech, Croatia, pp 543–574. https://doi.org/10.5772/5910

    Google Scholar 

  18. Intan SR, Haslina A, Yahaya NHM, Sulong AB (2010) Development of Visualization Application (VJBK) for Newly Designed Jig and Fixture for Computer-Assisted Knee Replacement Surgery. In: International Business Information Management Conference (14th IBIMA), Istanbul Turkey, 2010. pp 882-890. http://www.ibima.org/TR2010/papers/int.html

  19. Jacoby RH, Ferneau M, Humphries J (1994) Gestural interaction in a virtual environment. Proceedings of SPIE 1994:355–364. https://doi.org/10.1117/12.173892

    Article  Google Scholar 

  20. Jauregui DAG, Argelaguet F, Lecuyer A (2012) Design and evaluation of 3D cursors and motion parallax for the exploration of desktop virtual environments. In: 2012 I.E. Symposium on 3D User Interfaces (3DUI), 4-5 March 2012 2012. pp 69-76. https://doi.org/10.1109/3dui.2012.6184186

  21. Jiang H, Kessler GD, Nonnemaker J (2002) DEMIS: a dynamic event model for interactive systems. In: Proceedings of the ACM symposium on Virtual reality software and technology, Hong Kong, China, 2002. ACM, 585757, pp 97-104. https://doi.org/10.1145/585740.585757

  22. Kadri A, Lecuyer A, Burkhardt JM (2007) The Visual Appearance of User's Avatar Can Influence the Manipulation of Both Real Devices and Virtual Objects. In: 3DUI '07. IEEE Symposium on 3D User Interfaces, 2007., 10-11 March 2007 2007. pp -. https://doi.org/10.1109/3dui.2007.340767

  23. Kavakli M, Jayarathna D (2005) Virtual Hand: An Interface for Interactive Sketching in Virtual Reality. In: International Conference on Computational Intelligence for Modelling, Control and Automation, International Conference on Intelligent Agents, Web Technologies and Internet Commerce., 28-30 Nov. 2005 2005. pp 613–618. https://doi.org/10.1109/CIMCA.2005.1631331

  24. Köpsel A, Majaranta P, Isokoski P, Huckauf A (2016) Effects of auditory, haptic and visual feedback on performing gestures by gaze or by hand. Behav Inform Technol 35(12):1044–1062. https://doi.org/10.1080/0144929X.2016.1194477

  25. Kosch T, Kettner R, Funk M, Schmidt A (2016) Comparing Tactile, Auditory, and Visual Assembly Error-Feedback for Workers with Cognitive Impairments. In: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, Reno, Nevada, USA, 2016. ACM, 2982157, pp 53-60. https://doi.org/10.1145/2982142.2982157

  26. Lam MC, Arshad H (2011) Virtual Hand Modeling and Simulation Based on Unity 3D. International Review on Computers and Software 6 (6):1044-1049. http://connection.ebscohost.com/c/articles/89747281/virtual-hand-modeling-simulation-based-unity-3d

  27. Liang J, Green M (1994) JDCAD: A highly interactive 3D modeling system. Computers & Graphics - CG 18(4):499–506. https://doi.org/10.1016/0097-8493(94)90062-0

    Article  Google Scholar 

  28. Mine M (1995) Virtual Environment Interaction Techniques (trans: Science DoC). Mine95virtualenvironment. University of North Carolina at Chapel Hill Chapel Hill, NC, USA. http://dl.acm.org/citation.cfm?id=897820

  29. Nguyen TTH, Duval T, Pontonnier C (2014) A new direct manipulation technique for immersive 3d virtual environments. In: ICAT-EGVE 2014: the 24th International Conference on Artificial Reality and Telexistence and the 19th Eurographics Symposium on Virtual Environments, 2014. p 8

  30. Olwal A, Feiner S (2003) The Flexible Pointer: An Interaction Technique for Augmented and Virtual Reality. In: Proceedings of ACM Symposium on User Interface Software and Technology, 2003. pp 81-82. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.214.629

  31. Ortega M (2013) 3D object position using automatic viewpoint transitions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris, France, 2013. ACM, 2470681, pp 193-196. https://doi.org/10.1145/2470654.2470681

  32. Osawa N (2006) Automatic adjustments for efficient and precise positioning and release of virtual objects. In: Proceedings of the 2006 ACM international conference on Virtual reality continuum and its applications, Hong Kong, China, 2006. ACM, 1128943, pp 121-128. https://doi.org/10.1145/1128923.1128943

  33. Osawa N, Asai K (2010) Adjustment and control methods for precise rotation and positioning of virtual object by hand. In: Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and its Applications in Industry, Seoul, South Korea, 2010. ACM, 1900205, pp 131-138. https://doi.org/10.1145/1900179.1900205

  34. Petzold B, Zaeh MF, Faerber B, Deml B, Egermeier H, Schilp J, Clarke S (2004) A Study on Visual, Auditory, and Haptic Feedback for Assembly Tasks. Presence: Teleoperators and Virtual Environments 13(1):16–21. https://doi.org/10.1162/105474604774048207

    Article  Google Scholar 

  35. Pierce JS, Forsberg AS, Conway MJ, Hong S, Zeleznik RC, Mine MR (1997) Image plane interaction techniques in 3D immersive environments. In: Proceedings of the 1997 symposium on Interactive 3D graphics, Providence, Rhode Island, United States, 1997. ACM, 253303, pp 39-ff. https://doi.org/10.1145/253284.253303

  36. Poupyrev I, Billinghurst M, Weghorst S, Ichikawa T (1996) The go-go interaction technique: non-linear mapping for direct manipulation in VR. In: Proceedings of the 9th annual ACM symposium on User interface software and technology, Seattle, Washington, United States, 1996. ACM, 237102, pp 79-80. https://doi.org/10.1145/237091.237102

  37. Poupyrev I, Weghorst S, Billinghurst M, Ichikawa T (1998) A study of techniques for selecting and positioning objects in immersive VEs: effects of distance, size, and visual feedback. In: Proceedings of ACM CHt'98, 1998. pp -. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.34.1169

  38. Prachyabrued M, Borst CW (2012) Visual interpenetration tradeoffs in whole-hand virtual grasping. In: 3D User Interfaces (3DUI), 2012 I.E. Symposium on, 2012. IEEE, pp 39-42

  39. Prachyabrued M, Borst CW (2012) Virtual grasp release method and evaluation. International Journal of Human-Computer Studies 70(11):828–848. https://doi.org/10.1016/j.ijhcs.2012.06.002

    Article  Google Scholar 

  40. Prachyabrued M, Borst CW (2014) Visual feedback for virtual grasping. In: 3D User Interfaces (3DUI), 2014 I.E. Symposium on, 2014. IEEE, pp 19-26

  41. Smith G, Salzman T, Stuerzlinger W (2001) 3D scene manipulation with 2D devices and constraints. In: Graphics interface 2001, Ottawa, Ontario, Canada, 2001. Canadian Information Processing Society, 781003, pp 135-142. http://dl.acm.org/citation.cfm?id=781003

  42. Stuerzlinger W, Wingrave C (2011) The Value of Constraints for 3D User Interfaces. In: Brunnett G, Coquillart S, Welch G (eds) Virtual Realities. Springer Vienna, pp 203-223. https://doi.org/10.1007/978-3-211-99178-7_11

  43. Teather RJ, Stuerzlinger W (2014) Visual aids in 3D point selection experiments. In: Proceedings of the 2nd ACM symposium on Spatial user interaction, 2014. ACM, pp 127-136

  44. Vanacken L, Grossman T, Coninx K (2009) Multimodal selection techniques for dense and occluded 3D virtual environments. International Journal of Human-Computer Studies 67(3):237–255. https://doi.org/10.1016/j.ijhcs.2008.09.001

    Article  Google Scholar 

  45. Viciana-Abad R, Reyes-Lecuona A, Rosa-Pujazón A, Pérez-Lorenzo JM (2014) The influence of different sensory cues as selection feedback and co-location in presence and task performance. Multimedia tools and applications 68(3):623–639

    Article  Google Scholar 

  46. Wan Hg, Chen Ff, Han Xx (2009) A 4-layer flexible virtual hand model for haptic interaction. In: VECIMS '09. IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurements Systems, 2009, 11-13 May 2009 2009. pp 185-190. https://doi.org/10.1109/VECIMS.2009.5068890

  47. Wilkes C, Bowman DA (2008) Advantages of velocity-based scaling for distant 3D manipulation. In: Proceedings of the 2008 ACM symposium on Virtual reality software and technology, Bordeaux, France, 2008. ACM, 1450585, pp 23-29. https://doi.org/10.1145/1450579.1450585

  48. Wu C-H, Chen W-L, Lin CH (2015) Depth-based hand gesture recognition. Multimedia Tools and Applications 75(12):7065–7086. https://doi.org/10.1007/s11042-015-2632-3

  49. Wyss HP, Blach R, Bues M (2006) ISith - Intersection-based spatial interaction for two hands. In: IEEE Symposium on 3D User Interfaces, 2006. pp 59-61. https://doi.org/10.1109/VR.2006.93

Download references

Acknowledgements

This research work was supported by the AP-2013-011 and the GGPM-2015-023 from Universiti Kebangsaan Malaysia (UKM) Grant Scheme. Other than that, the authors would like to thank the members of MyXLab, the Faculty of Information Science & Technology, Universiti Kebangsaan Malaysia for their support in this research.

Author information

Authors and Affiliations

  1. Center for Artificial Intelligence Technology, Faculty of Information Science & Technology, Universiti Kebangsaan Malaysia, 43600, Bangi, Selangor, Malaysia

    Meng Chun Lam, Haslina Arshad & Siok Yee Tan

  2. Department of Information Technology, Faculty of Computing and Information Technology, Rabigh King Abdulaziz University, Rabigh, 21911, Saudi Arabia

    Anton Satria Prabuwono

  3. Dana Research Center, Northeastern University, Boston, MA, USA

    S. M. M. Kahaki

Authors
  1. Meng Chun Lam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haslina Arshad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anton Satria Prabuwono
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Siok Yee Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. S. M. M. Kahaki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meng Chun Lam.

Ethics declarations

Conflict of interests

The author declares that there is no conflict of interests regarding the publication of this paper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lam, M.C., Arshad, H., Prabuwono, A.S. et al. Interaction techniques in desktop virtual environment: the study of visual feedback and precise manipulation method. Multimed Tools Appl 77, 16367–16398 (2018). https://doi.org/10.1007/s11042-017-5205-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5205-9

Keywords

Search

Navigation