Abstract
We introduce FlowMouse, a computer vision-based pointing device and gesture input system. FlowMouse uses optical flow techniques to model the motion of the hand and a capacitive touch sensor to enable and disable interaction. By using optical flow rather than a more traditional tracking based method, FlowMouse is exceptionally robust, simple in design, and offers opportunities for fluid gesture-based interaction that go well beyond merely emulating pointing devices such as the mouse. We present a Fitts law study examining pointing performance, and discuss applications of the optical flow field for gesture input.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Anandan, P.: A Computational Framework and Algorithm for the Measurement of Visual Motion. International Journal of Computer Vision 2, 283–310 (1989)
Barron, J., Fleet, D., Beauchemin, S., Burkitt, T.: Performance of Optical Flow Techniques. In: Computer Vision and Pattern Recognition (1992)
Bathiche, S.: Pointer Ballistics for Windows XP (2002), http://www.microsoft.com/whdc/device/input/pointer-bal.mspx
Bradski, G., Eruhimov, V., Molinov, S., Mosyagin, V., Pisarevsky, V.: A Video Joystick from a Toy. In: Proceedings of the 2001 Workshop on Perceptive User Interfaces (2001)
Buxton, W.: A Three-State Model of Graphical Input. In: INTERACT 1990 (1990)
Buxton, W., Meyers, B.: A Study in Two-Handed Input. In: Proc. of CHI 1986: ACM Conference on Human Factors in Computing Systems (1986)
Cao, X., Balakrishnan, R.: VisionWand: Interaction Techniques for Large Displays Using a Passive Wand Tracked in 3D. In: ACM Symposium on User Interface Software and Technology (2003)
Cutler, R., Turk, M.: View-based Interpretation of Real-time Optical Flow for Gesture Recognition. In: IEEE Conference on Automatic Face and Gesture Recognition (1998)
Douglas, S., Kirkpatrick, A., MacKenzie, I.S.: Testing Pointing Device Performance and User Assessment with the ISO 9241, Part 9 Standard. In: Proc. CHI 1999 (1999)
Fitzmaurice, G.W., Ishii, H., Buxton, W.: Bricks: Laying the Foundations for Graspable User Interfaces. In: Proceedings of CHI 1995 (1995)
Hinckley, K., Sinclair, M.: Touch-Sensing Input Devices. In: ACM CHI 1999 Conference on Human Factors in Computing Systems (1999)
Hinckley, K., Pausch, R., Proffitt, D., Kassell, N.: Interaction and Modeling Techniques for Desktop Two-Handed Input. In: ACM UIST 1998 Symposium on User Interface Software & Technology (1998)
Horn, B.K.P.: Closed Form Solution of Absolute Orientation Using Unit Quaternions. Journal of the Optical Society 4(4), 629–642 (1987)
Jellinek, H.D., Card, S.K.: Powermice and User Performance. In: SIGCHI Conference on Human Factors in Computing Systems (1990)
Kjeldsen, R., Kender, J.: Interaction with On-Screen Objects Using Visual Gesture Recogntion. In: CVPR 1997 (1997)
Krueger, M.: Artificial Reality II. Addison-Wesley, Reading (1991)
Kurtenbach, G., Buxton, W.: The Limits of Expert Performance Using Hierarchic Marking Menus. In: Proceedings of InterCHI 1999 (1993)
Lenman, S., Bretzner, L., Thuresson, B.: Using Marking Menus to Develop Command Sets for Computer Vision Based Hand Gesture Interfaces. In: Proceedings of the Second Nordic Conference On Human-Computer Interaction (2002)
Letessier, J., Berard, F.: Visual Tracking of Bare Fingers for Interactive Surfaces. In: ACM Symposium on User Interface Software and Technology (2004)
MacKenize, I.S.: Fitts’ Law as Research and Design Tool in Human-Computer Interaction. In: Human-Computer Interaction 1992, pp. 91–139 (1992)
Quek, F., Mysliwiec, T., Zhao, M.: FingerMouse: A Freehand Computer Pointing Interface. In: Proc. of Int’l Conf. on Automatic Face and Gesture Recognition (1995)
Rekimoto, J.: ThumbSense: Automatic Mode Sensing for Touchpad-based Interactions. In: CHI 2003 Late Breaking Results (2003)
Rekimoto, J., Ayatsuka, Y.: CyberCode: Designing Augmented Reality Environments with Visual Tags. In: Designing Augmented Reality Environments, DARE 2000 (2000)
Turk, M., Robertson, G.: Perceptual User Interfaces. Communications of the ACM (2000)
Wellner, P.: Interacting with Paper on the DigitalDesk. Communications of the ACM 36(7), 86–97 (1993)
Wilson, A.: TouchLight: An Imaging Touch Screen and Display for Gesture-Based Interaction. In: International Conference on Multimodal Interfaces (2004)
Wilson, A., Oliver, N.: GWindows: Towards Robust Perception-Based UI. In: First IEEE Workshop on Computer Vision and Pattern Recognition for Human Computer Interaction (2003)
Wu, M., Balakrishnan, R.: Multi-finger and Whole Hand Gestural Interaction Techniques for Multi-User Tabletop Displays. In: ACM Symposium on User Interface Software and Technology (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 IFIP International Federation for Information Processing
About this paper
Cite this paper
Wilson, A.D., Cutrell, E. (2005). FlowMouse: A Computer Vision-Based Pointing and Gesture Input Device. In: Costabile, M.F., Paternò, F. (eds) Human-Computer Interaction - INTERACT 2005. INTERACT 2005. Lecture Notes in Computer Science, vol 3585. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11555261_46
Download citation
DOI: https://doi.org/10.1007/11555261_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28943-2
Online ISBN: 978-3-540-31722-7
eBook Packages: Computer ScienceComputer Science (R0)