Abstract
Emerging above-surface technology is an opportunity to exploit interaction spaces above a device’s surface; however, the resulting loss of the proprioceptive feedback available from on-surface interactions degrades the user’s sense of control and precision. We asked whether a pseudohaptic illusion (PHI) could help: a sense of force in the absence of actual contact, induced by manipulating the relation of body motion to graphical and auditory cues.
To examine the value of above-surface PHIs, we used a zooming microtask, because finger occlusion impedes current implementations on small displays such as smartwatches. In a qualitative study (N = 12), we were able to trigger a physical illusion most often described as elasticity in 92% of participants through physical control/graphical display (C/D) manipulation, and that audio cues significantly strengthened the illusion. Participants experiencing this PHI reported improved sense of control when zooming, and found the interaction’s physicality natural.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Agarawala, A., Balakrishnan, R.: Keepin’it real: pushing the desktop metaphor with physics, piles and the pen. In: Proceedings of CHI, pp. 1283–1292. ACM (2006)
Chen, X.A., Schwarz, J., Harrison, C., Mankoff, J., Hudson, S.E.: Air+touch: interweaving touch & in-air gestures. In: Proceedings of UIST, pp. 519–525. ACM (2014)
Colgan, A.: How does the leap motion controller work? http://blog.leapmotion.com/. Accessed 24 Mar 2017
Ernst, M.O., Bulthoff, H.H.: Merging the senses into a robust percept. Trends Cogn. Sci. 8(4), 162–169 (2004)
Goldstein, E.: Sensation and Perception, 5th edn. Wadsworth Publishing Company, Belmont (1999)
Han, J., Ahn, S., Lee, G.: Transture: continuing a touch gesture on a small screen into the air. In: CHI Extended Abstracts, pp. 1295–1300. ACM (2015)
Hayward, V.: A brief taxonomy of tactile illusions and demonstrations that can be done in a hardware store. Brain Res. Bull. 75(6), 742–752 (2008)
Kratz, S., Rohs, M.: Hoverflow: exploring around-device interaction with IR distance sensors. In: MobileHCI, pp. 42:1–42:4. ACM (2009)
Kuchenbecker, K.J., Fiene, J., Niemeyer, G.: Improving contact realism through event-based haptic feedback. IEEE Trans. Vis. Comp. Graph. 12(2), 219–230 (2006)
Leap Motion: Windows VR development (2017). https://developer.leapmotion.com/windows-vr. Accessed 28 Mar 2017
Lécuyer, A., Burkhardt, J.M., Coquillart, S., Coiffet, P.: Boundary of illusion: an experiment of sensory integration with a pseudo-haptic system. In: IEEE VR, pp. 115–122 (2001)
Lécuyer, A., Burkhardt, J.M., Etienne, L.: Feeling bumps and holes without a haptic interface: the perception of pseudo-haptic textures. In: Proceedings of CHI, pp. 239–246. ACM (2004)
Lee, J., Baek, S.: Elastic cursor and elastic edge: applying simulated resistance to interface elements for seamless edge-scroll. In: Adjunct Proceedings of UIST, pp. 63–64. ACM (2015)
MacLean, K.E., Schneider, O., Seifi, H.: The diverse tasks of haptic and multimodal design and how to support them. In: Handbook of Multimodal-Multisensor Interfaces, pp. 97–142. Morgan Claypool (2017)
Mandryk, R.L., Rodgers, M.E., Inkpen, K.M.: Sticky widgets: pseudo-haptic widget enhancements for multi-monitor displays. In: CHI Extended Abstracts, pp. 1621–1624. ACM (2005)
Marquardt, N., Jota, R., Greenberg, S., Jorge, J.A.: The continuous interaction space: interaction techniques unifying touch and gesture on and above a digital surface. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011. LNCS, vol. 6948, pp. 461–476. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23765-2_32
Mine, M.R., Brooks Jr., F.P., Sequin, C.H.: Moving objects in space: exploiting proprioception in virtual-environment interaction. In: SIGGRAPH, pp. 19–26. ACM (1997)
Nancel, M., Wagner, J., Pietriga, E., Chapuis, O., Mackay, W.: Mid-air pan-and-zoom on wall-sized displays. In: Proceedings of CHI, pp. 177–186. ACM (2011)
Pusch, A., Lécuyer, A.: Pseudo-haptics: from the theoretical foundations to practical system design guidelines. In: International Conference on Multimodal Interfaces, pp. 57–64. ACM (2011)
Sarwar, M.S., Dobashi, Y., Preston, C., et al.: Bend, stretch, and touch: locating a finger on an actively deformed transparent sensor array. Sci. Adv. 3(3), e1602200 (2017)
Sridhar, S., Markussen, A., et al.: Watchsense: on-and above-skin input sensing through a wearable depth sensor. In: Proceedings of CHI, pp. 3891–3902. ACM (2017)
Srinivasan, M.A., Beauregard, G.L., Brock, D.: The impact of visual information on the haptic perception of stiffness in virtual environments. In: Haptic Symposium, vol. DSC:58, pp. 555–559. ASME (1996)
Wu, W.C., Basdogan, C., Srinivasan, M.A.: Visual, haptic and bimodal perception of size and stiffness in virtual environments. In: Haptic Symposium, vol. DSC:67, pp. 19–26. ASME (1999)
Acknowledgments
This study was funded by Qualcomm and NSERC, carried out under UBC Ethics #H16-01549. We thank our UBC ECE collaborators Prof. John Madden and Mirza Saquib, and our many helpful colleagues in SPIN lab and the Designing for People research network.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Ustek, D., Chow, K., Zhang, H., MacLean, K. (2018). A Multimodal Illusion of Force Improves Control Perception in Above-Surface Gesture: Elastic Zed-Zoom. In: Prattichizzo, D., Shinoda, H., Tan, H., Ruffaldi, E., Frisoli, A. (eds) Haptics: Science, Technology, and Applications. EuroHaptics 2018. Lecture Notes in Computer Science(), vol 10893. Springer, Cham. https://doi.org/10.1007/978-3-319-93445-7_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-93445-7_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93444-0
Online ISBN: 978-3-319-93445-7
eBook Packages: Computer ScienceComputer Science (R0)