Advertisement

SocioCon: A Social Circle for Your Interactive Devices

  • Ngoc Thi Nguyen
  • Hyowon Lee
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10919)

Abstract

Our everyday lives with technology are increasingly characterised by the use of multiple interactive devices. We are seeing more and more variety of specialised input and output devices that can connect with other devices to help the user perform a particular task in a more optimal or other beneficial way. When multiple devices are used together within a session, owing to the diversity of native interaction modalities, content representation and capabilities of each device, there are usability issues arising before, after or at the transition of inter-connectivity. In such a device-sharing situation, one device that could temporarily borrow the input or output modalities from other devices must know which devices are available or allowed to offer such connections, and how their combined use should push yet another way of end-user interactivity. In this paper, we envision a few typical usage scenarios where such multi-device interaction is expected or desirable, then from these develop an overarching concept of usability in a multi-device environment. Simple social heuristics in sharing the interactivity and resources amongst devices are employed to shape how the overall usability in these situations could be addressed and framed. We develop two multi-device interaction prototypes that correspond to some of these scenarios, conduct usability testing, report the findings and discuss further insights.

Keywords

Multi-devices Device-to-device Cross-devices Transition usability Input devices Output devices Context-aware Interaction design 

References

  1. 1.
    Jacob, R.J.K., Leggett, J.J., Myer, B.A., Pausch, R.: Interaction styles and input/output devices. Behav. Inf. Technol. 12(2), 69–79 (1993)CrossRefGoogle Scholar
  2. 2.
    Bolt, R.A.: “Put-that-there”: voice and gesture at the graphics interface. ACM SIGGRAPH Comput. Graph. 14(3), 262–270 (1980)CrossRefGoogle Scholar
  3. 3.
    Streitz, N.A., Geißler, J., Holmer, T., Konomi, S.I., Müller-Tomfelde, C., Reischl, W., Rexroth, P., Seitz, P., Steinmetz, R.: i-LAND: an interactive landscape for creativity and innovation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 120–127. ACM, Pittsburgh (1999)Google Scholar
  4. 4.
    Everitt, K., Shen, C., Ryall, K., Forlines, C.: MultiSpace: enabling electronic document micro-mobility in table-centric, multi-device environments. In: First IEEE International Workshop on Horizontal Interactive Human-Computer Systems, TableTop 2006. IEEE (2006)Google Scholar
  5. 5.
    Lee, H., Jeong, H., Lee, J.H., Yeom, K.-W., Shin, H.-J., Park, J.-H.: Select-and-point: a novel interface for multi-device connection and control based on simple hand gestures. In: CHI 2008 Extended Abstracts on Human Factors in Computing Systems, pp. 3357–3362. ACM, Florence (2008)Google Scholar
  6. 6.
    Rekimoto, J., Ayatsuka, Y., Kohno, M., Oba, H.: Proximal interactions: a direct manipulation technique for wireless networking. In: Interact 2003 (2003)Google Scholar
  7. 7.
    Greenberg, S., Boyle, M., LaBerge, J.: PDAs and shared public displays: making personal information public, and public information personal. Pers. Technol. 3(1), 54–64 (1999)CrossRefGoogle Scholar
  8. 8.
    Ballagas, R., Rohs, M., Sheridan, J.G.: Sweep and point and shoot: phonecam-based interactions for large public displays. In: CHI 2005 Extended Abstracts on Human Factors in Computing Systems. ACM (2005)Google Scholar
  9. 9.
    Cheverst, K., Dix, A., Fitton, D., Kray, C., Rouncefield, M., Sas, C., Saslis-Lagoudakis, G., Sheridan, J.G.: Exploring bluetooth based mobile phone interaction with the hermes photo display. In: Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices & Services. ACM (2005)Google Scholar
  10. 10.
    Hyakutake, A., Ozaki, K., Kitani, K.M., Koike, H.: 3-D interaction with a large wall display using transparent markers. In: Proceedings of the International Conference on Advanced Visual Interfaces. ACM (2010)Google Scholar
  11. 11.
    Rekimoto, J.: Gesturewrist and gesturepad: unobtrusive wearable interaction devices. In: Proceedings of Fifth International Symposium on Wearable Computers. IEEE (2001)Google Scholar
  12. 12.
    Morganti, E., Angelini, L., Adami, A., Lalanne, D., Lorenzelli, L., Mugellini, E.: A smart watch with embedded sensors to recognize objects, grasps and forearm gestures. Proc. Eng. 41, 1169–1175 (2012)CrossRefGoogle Scholar
  13. 13.
    Xu, C., Pathak, P.H., Mohapatra, P.: Finger-writing with Smartwatch, pp. 9–14 (2015)Google Scholar
  14. 14.
    Hinckley, K.: Synchronous gestures for multiple persons and computers. In: Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology. ACM (2003)Google Scholar
  15. 15.
    Hinckley, K., Ramos, G., Guimbretiere, F., Baudisch, P., Smith, M.: Stitching: pen gestures that span multiple displays. In: Proceedings of the Working Conference on Advanced Visual Interfaces. ACM (2004)Google Scholar
  16. 16.
    Microsoft Display Dock Overview - Microsoft - Global (2018). https://www.microsoft.com/en-ww/mobile/accessory/hd-500
  17. 17.
    Rekimoto, J.: Pick-and-drop: a direct manipulation technique for multiple computer environments. In: Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology, pp. 31–39. ACM, Banff (1997)Google Scholar
  18. 18.
    Rekimoto, J., Saitoh, M.: Augmented surfaces: a spatially continuous work space for hybrid computing environments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM (1999)Google Scholar
  19. 19.
    Lucero, A., Keränen, J., Korhonen, H.: Collaborative use of mobile phones for brainstorming. In: Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services, pp. 337–340. ACM, Lisbon (2010)Google Scholar
  20. 20.
    Brad, A., Myers, B.A.: Using handhelds and PCs together. Commun. ACM 44(11), 34–41 (2001)CrossRefGoogle Scholar
  21. 21.
    Robertson, S., Wharton, C., Ashworth, C., Franzke, M.: Dual device user interface design: PDAs and interactive television. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM (1996)Google Scholar
  22. 22.
    De Luca, A., Frauendienst, B.: A privacy-respectful input method for public terminals. In: Proceedings of the 5th Nordic Conference on Human-Computer Interaction: Building Bridges, pp. 455–458. ACM, Lund (2008)Google Scholar
  23. 23.
    Miyaoku, K., Higashino, S., Tonomura, Y.: C-blink: a hue-difference-based light signal marker for large screen interaction via any mobile terminal. In: Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology, pp. 147–156. ACM, Santa Fe (2004)Google Scholar
  24. 24.
    Geel, M., Huguenin, D., Norrie, M.C.: PresiShare: opportunistic sharing and presentation of content using public displays and QR codes. In: Proceedings of the 2nd ACM International Symposium on Pervasive Displays. ACM (2013)Google Scholar
  25. 25.
    Alt, F., Shirazi, A.S., Kubitza, T., Schmidt, A.: Interaction techniques for creating and exchanging content with public displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM (2013)Google Scholar
  26. 26.
    Chen, X.A., Grossman, T., Wigdor, D.J., Fitzmaurice, G.: Duet: exploring joint interactions on a smart phone and a smart watch. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 159–168. ACM, Toronto (2014)Google Scholar
  27. 27.
    Kubo, Y., Takada, R., Shizuki, B., Takahashi, S.: Exploring context-aware user interfaces for smartphone-smartwatch cross-device interaction. Proc. ACM Interact. Mob. Wearable Ubiquit. Technol. 1(3), 1–21 (2017)CrossRefGoogle Scholar
  28. 28.
    Noh, W., Lee, M., Cheon, H., Kim, J., Lee, K., Cho, J.: TakeOut: drawing application using distributed user interface for being close to real experience. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, pp. 173–176. ACM, Heidelberg (2016)Google Scholar
  29. 29.
    Trimeche, M., Suomela, R., Aaltonen, A., Lorho, G., Dossaji, T., Aarnio, T., Tuoriniemi, S.: Enhancing end-user experience in a multi-device ecosystem. In: Proceedings of the 4th International Conference on Mobile and Ubiquitous Multimedia, pp. 19–25. ACM, Christchurch (2005)Google Scholar
  30. 30.
    Li, J., Bu, Y., Chen, S., Tao, X., Lu, J.: FollowMe: on research of pluggable infrastructure for context-awareness. In: 20th International Conference on Advanced Information Networking and Applications, AINA 2006. IEEE (2006)Google Scholar
  31. 31.
    Johansson, D., Wiberg, M.: Conceptually advancing “Application Mobility” towards design: applying a concept-driven approach to the design of mobile IT for home care service groups. Int. J. Ambient Comput. Intell. (IJACI) 4(3), 20–32 (2012)CrossRefGoogle Scholar
  32. 32.
    Sørensen, H., Raptis, D., Kjeldskov, J., Skov, M.B.: The 4C framework: principles of interaction in digital ecosystems. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 87–97. ACM, Seattle (2014)Google Scholar
  33. 33.
    Turunen, M., Kallinen, A., Sànchez, I., Riekki, J., Hella, J., Olsson, T., Melto, A., Rajaniemi, J.P., Hakulinen, J., Mäkinen, E., Valkama, P.: Multimodal interaction with speech and physical touch interface in a media center application. In: Proceedings of the International Conference on Advances in Computer Entertainment Technology, pp. 19–26. ACM, Athens (2009)Google Scholar
  34. 34.
    Sørense, H., Kjeldskov, J.: The interaction space of a multi-device, multi-user music experience. In: Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design, pp. 504–513. ACM, Copenhagen (2012)Google Scholar
  35. 35.
    Jokela, T., Ojala, J., Olsson, T.: A diary study on combining multiple information devices in everyday activities and tasks. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 3903–3912. ACM, Seoul (2015)Google Scholar
  36. 36.
    Nguyen, N.T., Lee, H.: ‘Hop-to-select’ traverse with gestural input in an eye-off interaction. In: Proceedings of the 29th Australian Conference on Computer-Human Interaction, pp. 597–601. ACM, Brisbane (2017)Google Scholar
  37. 37.
    Nguyen, N.T., Nanayakkara, S., Lee, H.: Visual field visualizer: easier & scalable way to be aware of the visual field. In: Proceedings of the 9th Augmented Human International Conference, pp. 1–3. ACM, Seoul (2018)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Singapore University of Technology and DesignSingaporeSingapore

Personalised recommendations