Abstract
Interacting with digital contents in 3D is an essential task in various applications such as modeling packages, gaming, virtual reality, etc. Traditional interfaces using keyboard and mouse or trackball usually require a non-trivial amount of working space as well as a learning process. We present the design of EZ-Manipulator, a new 3D manipulation interface using smartphones that supports mobile, fast, and ambiguity-free interaction with 3D objects. Our system leverages the built-in multi-touch input and gyroscope sensor of smartphones to achieve 9 degrees-of-freedom axis-constrained manipulation and free-form rotation. Using EZ-Manipulator to manipulate objects in 3D is easy. The user merely has to perform intuitive singleor two-finger gestures and rotate the hand-held device to perform manipulations at fine-grained and coarse levels respectively.We further investigate the ambiguity in manipulation introduced by indirect manipulations using a multi-touch interface, and propose a dynamic virtual camera adjustment to effectively resolve the ambiguity. A preliminary study shows that our system has significant lower task completion time compared to conventional use of a keyboard–mouse interface, and provides a positive user experience to both novices and experts.
Article PDF
Similar content being viewed by others
References
Chen, M.; Mountford, S. J.; Sellen, A. A study in interactive 3-D rotation using 2-D control devices. In: Proceedings of the 15th Annual Conference on Computer Graphics and Interactive Techniques, 121–129, 1988.
Henriksen, K.; Sporring, J.; Hornbæk, K. Virtual trackballs revisited. IEEE Transactions on Visualization and Computer Graphics Vol. 10, No. (2), 206–216, 2004.
Bier, E. A. Skitters and jacks: Interactive 3D positioning tools. In: Proceedings of the 1986 Workshop on Interactive 3D graphics, 183–196, 1987.
Conner, B. D.; Snibbe, S. S.; Herndon, K. P.; Robbins, D. C.; Zeleznik, R. C.; van Dam, A. Threedimensional widgets. In: Proceedings of the Symposium on Interactive 3D Graphics, 183–188, 1992.
Ware, C.; Jessome, D. R. Using the bat: A six dimensional mouse for object placement. IEEE Computer Graphics and Applications Vol. 8, No. (6), 65–70, 1988.
Zimmerman, T. G.; Lanier, J.; Blanchard, C.; Bryson, S.; Harvill, Y. A hand gesture interface device. In: Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface, 189–192, 1986.
Froehlich, B.; Hochstrate, J.; Skuk, V.; Huckauf, A. The GlobeFish and the GlobeMouse: Two new six degree of freedom input devices for graphics applications. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 191–199, 2006.
Perelman, G.; Serrano, M.; Raynal, M.; Picard, C.; Derras, M.; Dubois, E. The roly-poly mouse: Designing a rolling input device unifying 2D and 3D interaction. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 327–336, 2015.
Wang, R. Y.; Popović, J. Real-time hand-tracking with a color glove. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 63, 2009.
Wang, R.; Paris, S.; Popović, J. 6D hands: Markerless hand-tracking for computer aided design. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, 549–558, 2011.
Song, P.; Goh, W. B.; Hutama, W.; Fu, C.-W.; Liu, X. A handle bar metaphor for virtual object manipulation with mid-air interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1297–1306, 2012.
Hancock, M.; Carpendale, S.; Cockburn, A. Shallowdepth 3d interaction: Design and evaluation of one-, two- and three-touch techniques. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1147–1156, 2007.
Hancock, M.; ten Cate, T.; Carpendale, S. Sticky tools: Full 6DOF force-based interaction for multi-touch tables. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, 133–140, 2009.
Reisman, J. L.; Davidson, P. L.; Han, J. Y. A screenspace formulation for 2D and 3D direct manipulation. In: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, 69–78, 2009.
Martinet, A.; Casiez, G.; Grisoni, L. The effect of DOF separation in 3D manipulation tasks with multi-touch displays. In: Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology, 111–118, 2010.
Martinet, A.; Casiez, G.; Grisoni, L. Integrality and separability of multitouch interaction techniques in 3D manipulation tasks. IEEE Transactions on Visualization and Computer Graphics Vol. 18, No. (3), 369–380, 2012.
Liu, J.; Au, O.K.-C.; Fu, H.; Tai, C.-L. Two-finger gestures for 6DOF manipulation of 3D objects. Computer Graphics Forum Vol. 31, No. (7), 2047–2055, 2012.
Au, O.K.-C.; Tai, C.-L.; Fu, H. Multitouch gestures for constrained transformation of 3D objects. Computer Graphics Forum Vol. 31, No. 2pt3, 651–660, 2012.
Hachet, M.; Bossavit, B.; Cohé, A.; de la Rivière, J.-B. Toucheo: Multitouch and stereo combined in a seamless workspace. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, 587–592, 2011.
Schmidt, R.; Singh, K.; Balakrishnan, R. Sketching and composing widgets for 3D manipulation. Computer Graphics Forum Vol. 27, No. (2), 301–310, 2008.
Cohé, A.; Décle, F.; Hachet, M. tBox: A 3d transformation widget designed for touch-screens. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 3005–3008, 2011.
Kin, K.; Miller, T.; Bollensdorff, B.; DeRose, T.; Hartmann, B.; Agrawala, M. Eden: A professional multitouch tool for constructing virtual organic environments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1343–1352, 2011.
Sun, Q.; Lin, J.; Fu, C.-W.; Kaijima, S.; He, Y. A multi-touch interface for fast architectural sketching and massing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 247–256, 2013.
Paczkowski, P.; Dorsey, J.; Rushmeier, H.; Kim, M. H. Paper3D: Bringing casual 3D modeling to a multi-touch interface. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, 23–32, 2014.
Chen, C.-Y.; Yao, C.-Y.; Lin, C.-Y.; Hung, S.-H. Realtime remote control of artificial cilia actuation using fingertip drawing for efficient micromixing. Journal of Laboratory Automation Vol. 19, No. (5), 492–497, 2014.
Hilliges, O.; Kim, D.; Izadi, S.; Weiss, M.; Wilson, A. HoloDesk: Direct 3d interactions with a situated see-through display. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2421–2430, 2012.
Nancel, M.; Wagner, J.; Pietriga, E.; Chapuis, O.; Mackay, W. Mid-air pan-and-zoom on wall-sized displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 177–186, 2011.
Microsoft. Kinect for Xbox360. 2010. Available at https://www.xbox.com/enUS/kinect.
Chapuis, O.; Bezerianos, A.; Frantzeskakis, S. Smarties: An input system for wall display development. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2763–2772, 2014.
Song, J.; Sörös, G.; Pece, F.; Fanello, S. R.; Izadi, S.; Keskin, C.; Hilliges, O. In-air gestures around unmodified mobile devices. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, 319–329, 2014.
Boring, S.; Gehring, S.; Wiethoff, A.; Blöckner, A. M.; Schöning, J.; Butz, A. Multi-user interaction on media facades through live video on mobile devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2721–2724, 2011.
Boring, S.; Baur, D.; Butz, A.; Gustafson, S.; Baudisch, P. Touch projector: Mobile interaction through video. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2287–2296, 2010.
Chang, Y.-C.; Wang, H.-C.; Chu, H.-K.; Lin, S.-T.; Wang, S.-P. AlphaRead: Support unambiguous referencing in remote collaboration with readable object annotation. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work and Social Computing, 2246–2259, 2017.
Kashiwakuma, J.; Kitahara, I.; Kameda, Y.; Ohta, Y. A virtual camera controlling method using multitouch gestures for capturing free-viewpoint video. In: Proceedings of the 11th European Conference on Interactive TV and Video, 67–74, 2013.
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Po-Huan Tseng received his master degree from “National Tsing Hua University”, Taiwan, China. He is currently working at Synology as a software engineer. His research interests include computer graphics and human–computer interaction.
Shih-Hsuan Hung is a research assistant in the GAME center in “National Taiwan University of Science and Technology” (NTUST), Taipei, Taiwan, China. He received his M.S. degree in computer science and information engineering (CSIE) from NTUST in 2016. His research interests are in computer graphics and multimedia.
Pei-Ying Chiang is an assistant professor in CSIE at “National Taipei University of Technology”, Taipei, Taiwan, China. She received her M.S. and Ph.D. degrees in compucter science from University of Southern California in 2006 and 2011 respectively. Her research interests are in computer graphics and human–computer interaction.
Chih-Yuan Yao is an associate professor in CSIE at NTUST, Taipei, Taiwan, China. He received his M.S. and Ph.D. degrees in CSIE from “National Cheng Kung University”, Tainan, Taiwan, China, in 2003 and 2010 respectively. His research interest are in computer graphics and multimedia.
Hung-Kuo Chu received his Ph.D. degree from the Department of Computer Science and Information Engineering, “National Cheng Kung University”, in 2010. He is currently an associate professor in the Department of Computer Science, “National Tsing Hua University”. His research interests focus on shape understanding, smart manipulation, perception-based rendering, recreational graphics, and human–computer interaction.
Electronic supplementary material
EZ-Manipulator Designing a Mobile, Fast, and Ambiguity-Free 3D Manipulation Interface using Smartphones
Rights and permissions
Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Tseng, PH., Hung, SH., Chiang, PY. et al. EZ-Manipulator: Designing a mobile, fast, and ambiguity-free 3D manipulation interface using smartphones. Comp. Visual Media 4, 139–147 (2018). https://doi.org/10.1007/s41095-018-0105-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-018-0105-0