Abstract
In this paper, we propose an intuitive pointing position estimation method for large scale display interaction in top-view depth images. The depth sensor is mounted above the users’ head in order to avoid the sensor occluding the display. In order to estimate the pointing position, we detect the user’s head and estimate the position of the user’s eye. To calculate the center of the head, we propose a head segmentation method. We use an iterative binary partitioning method and a one-to-one correspondence method to detect and track the hands, respectively. The 3D positions of the head and hands were converted to the real world coordinates and the pointing position was estimated on the eye-hand ray intersecting with the large screen. Experimental results show that we improve the head detection rate applying our head segmentation method. Also, we calculate the pointing direction accuracy and the proposed method has a good performance compared with conventional methods even in dark environments.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ball, R., North, C., Bowman, D.A.: Move to improve: promoting physical navigation to increase user performance with large displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 191–200 (2007)
Hu, K., Canavan, S., Yin, L.: Hand pointing estimation for human computer interaction based on two orthogonal-views. In: International Conference on Pattern Recognition, pp. 3760–3763. IEEE (2010)
Jojic, N., Brumitt, B., Meyers, B., Harris, S., Huang, T.: Detection and estimation of pointing gestures in dense disparity maps. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 468–475 (2000)
Kim, J.S., Park, J.M.: Sensor-display registration for 3D physical user interaction using a flat-panel display. In: International Conference on Pattern Recognition, pp. 1675–1680 (2014)
Lee, D., Lee, S.: Vision-based finger action recognition by angle detection and contour analysis. ETRI J. 33, 415–422 (2011)
Nickel, K., Stiefelhagen, R.: Pointing gesture recognition based on 3D-tracking of face, hands and head orientation. In: Proceedings of the 5th International Conference on Multimodal Interfaces, pp. 140–146 (2003)
Park, C.B., Lee, S.W.: Real-time 3D pointing gesture recognition for mobile robots with cascade HMM and particle filter. Image Vis. Comput. 29, 51–63 (2011)
Pateraki, M., Baltzakis, H., Trahanias, P.: Visual estimation of pointed targets for robot guidance via fusion of face pose and hand orientation. Comput. Vis. Image Underst. 120, 1–13 (2014)
Richarz, J., Scheidig, A., Martin, C., Muller, S., Gross, H.M.: A monocular pointing pose estimator for gestural instruction of a mobile robot. Int. J. Adv. Robot. Syst. 4, 139–150 (2007)
Schick, A., van de Camp, F., Ijsselmuiden, J., Stiefelhagen, R.: Extending touch: towards interaction with large-scale surfaces. In: Proceedings of the ACM International Conference on Interactive Tabletops And Surfaces, pp. 117–124 (2009)
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kip-man, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1297–1304. IEEE (2011)
Tang, S., Wang, X., Lv, X., Han, T.X., Keller, J., He, Z., Skubic, M., Lao, S.: Histogram of oriented normal vectors for object recognition with a depth sensor. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7725, pp. 525–538. Springer, Heidelberg (2013). doi:10.1007/978-3-642-37444-9_41
Taylor, J.L., McCloskey, D.: Pointing. Behav. Brain Res. 29, 1–5 (1988)
Vogel, D., Balakrishnan, R.: Distant freehand pointing and clicking on very large, high resolution displays. In: Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, pp. 33–42 (2005)
Yoo, B., Han, J.J., Choi, C., Yi, K., Suh, S., Park, D., Kim, C.: 3D user interface combining gaze and hand gestures for large-scale display. In: CHI 2010 Extended Abstracts on Human Factors in Computing Systems, pp. 3709–3714 (2010)
Acknowledgement
This work was supported by the ICT R&D program of MSIP/IITP. (15501-15-1016, Instant 3D object-based Join & Joy content technology supporting simultaneous participation of users in remote places and enabling realistic experience).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Kim, Hm., Kim, D., Kim, Y.S., Kim, KH. (2017). Intuitive Pointing Position Estimation for Large Scale Display Interaction in Top-View Depth Images. In: Chen, CS., Lu, J., Ma, KK. (eds) Computer Vision – ACCV 2016 Workshops. ACCV 2016. Lecture Notes in Computer Science(), vol 10118. Springer, Cham. https://doi.org/10.1007/978-3-319-54526-4_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-54526-4_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-54525-7
Online ISBN: 978-3-319-54526-4
eBook Packages: Computer ScienceComputer Science (R0)