Skip to main content
SpringerLink
Log in

Vision based multi-user human computer interaction

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a vision-based Multi-user Human Computer Interaction (HCI) method for creating augmented reality user interfaces. In the HCI session, one of the users’ hands is selected as the active hand. The fingers of the active hand are employed as input devices to trigger functionalities of the application program. To share the token of interaction among the users, the HCI session is modeled as a Finite State Machine (FSM). The FSM is composed of the initial and steady states. In the initial state, the FSM identifies the active hand by tracking the hand with the maximum moving speed. Then the FSM enters the steady state to carry out the HCI session. At the end of each individual HCI cycle, the FSM polls requests from other hands for acquiring the role of active hand. If such requests are sensed, the FSM returns to the initial state to track a new active hand. Otherwise, the HCI session is continuously carried out by the current active hand. Test results show that the resultant user interface is efficient, flexible and practical for users with problems on using ordinary input devices. In a desk-top computer equipped with a 640 × 480 resolution web-camera, the HCI session can be successfully conducted when the operation distance ranges from 30 to 90 cm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Azuma RT (1997) A survey of augmented reality. Presence: Teleoperators and Virtual Environments 6(4):355–385

    Article  Google Scholar 

  2. Erol A, Bebis G, Nicolescu M, Boyle RD, Twombly X (2007) Vision-based hand pose estimation: a review. Comput Vis Image Underst 108(1):52–73

    Article  Google Scholar 

  3. Gutzeit E, Vahl M, Zhou Z, von Lukas U (2011) Skin cluster tracking and verification for hand gesture recognition. In Proceedings of ISPA 2011, pp. 241–246

  4. Hasan MM, Mishra P (2012) Hand gesture modeling and recognition using geometric features: a review. Canadian Journal on Image Processing and Computer Vision 3(1):12–26

    Google Scholar 

  5. http://www.engadget.com/2014/10/08/kinect-for-windows-finger-tracking/, read 2015/09/28.

  6. https://dev.windows.com/en-us/kinect/hardware, read 2015/09/28.

  7. https://developer.leapmotion.com/documentation/java/devguide/Leap_Coordinate_Mapping.html, read 2015/08/10.

  8. https://msdn.microsoft.com/zh-tw/library/hh438998.aspx, read 2015/08/10.

  9. https://www.asus.com/tw/3D-Sensor/Xtion_PRO_LIVE/specifications/, read 2015/08/10.

  10. Lee B, Chun J (2009) Manipulation of virtual objects in marker-less AR system by fingertip tracking and hand gesture recognition. In Proceedings of ICIS 2009, pp. 1110–1115

  11. Lee D, Lee SG (2011) Vision-based finger action recognition by angle detection and contour analysis. ETRI J 33(3):415–422

    Article  Google Scholar 

  12. Letessier J, Berard F (2004) Visual tracking of bare fingers for interactive surfaces. In Proceedings of ACM UIST’04, pp. 119–122

  13. Manresa C, Varona J, Mas R, Perales FJ (2005) Real-time hand tracking and gesture recognition for human-computer interaction. Electronic Letters on Computer Vision and Image Analysis 5(3):96–104

    Google Scholar 

  14. Kay Elemetric Corporation (2003) Multi-Dimensional Voice Program (MDVP) operations manual, Model 5105

  15. Raheja JL, Das K, Chaudhary A (2011) An efficient real time method of fingertip detection. In Proceedings of TIMA-2011, pp. 447–450

  16. Rekimoto J (2002) SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In Proceedings of ACM CHI 2002, pp. 113–120

  17. Ueng SK, Luo CM, Chang H (2012) Voice quality assessment and visualization. In Proceeding of CISIS 2012, pp. 618–623

  18. Wilson AD (2010) Using a depth camera as a touch sensor. In Proceedings of ITS2010: Devices and Algorithms, pp. 69–72

  19. Wu YW, Ai XY (2008) Face detection in color images using AdaBoost algorithm based on skin color information. In Proceedings of IEEE Workshop on Knowledge Discovery and Data Mining, pp. 339–342

  20. Zhou H (2009) An OPS hand tracking algorithm based on optical flow. In Proceedings of World Congress on Software Engineering, pp. 190–194

  21. Zhou H, Ruan Q, Chen H (2010) A new approach of hand tracking based on integrated optical flow analysis. In Proceedings of ICSP 2010, pp. 1194–1197

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, National Taiwan Ocean University, Keelung City, 202, Taiwan

    Shyh-Kuang Ueng & Guan-Zhi Chen

Authors
  1. Shyh-Kuang Ueng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guan-Zhi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shyh-Kuang Ueng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ueng, SK., Chen, GZ. Vision based multi-user human computer interaction. Multimed Tools Appl 75, 10059–10076 (2016). https://doi.org/10.1007/s11042-015-3061-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3061-z

Keywords

Search

Navigation