Device Transition: Understanding Usability Issues in Shifting a Device During a Task

  • Ngoc Thi NguyenEmail author
  • Hyowon Lee
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11584)


Solutions for enhancing user experience in engaging multiple devices for a task largely imply a tight coupling between device combinations and their supporting user interface (UI) and interaction, thus usability issues may arise when end-users create own combinations of devices not foreseen by designers or developers. We propose the three design principles that foster spontaneous shifts in device engagement: partnership discoverability, role election and UI-interaction election. These principles are examined and realized through shifting cues existed in pre-transition, transition and post-transition phases of the transition pathway. Designed as independent user interfaces, shifting cues give hints to users about available nearby devices and guide the shifts in device engagement. Revisiting the design principles and know-how–so far accumulated based on the single device interaction–will be an important step towards realizing a usable interaction design that considers the increasingly common situations of using and shifting around among multiple devices while conducting a task.


Multi-device interaction Device composite shift Shifting cues Transition pathway Spontaneous device shift 


  1. 1.
    Sharp, H., Rogers, Y., Preece, J.: Interaction Design: Beyond Human Computer Interaction. Wiley, Hoboken (2007)Google Scholar
  2. 2.
    Denis, C., Karsenty, L.: Inter-usability of multi-device systems–a conceptual framework. Multiple User Interfaces: Cross-Platform Applications and Context-Aware Interfaces, pp. 373–385 (2004)CrossRefGoogle Scholar
  3. 3.
    Sørensen, H., Raptis, D., Kjeldskov, J., Skov, M.B.: The 4C framework: principles of interaction in digital ecosystems. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 87–97. ACM, Seattle (2014)Google Scholar
  4. 4.
    Bales, E., Sohn, T., Setlur, V.: Planning, apps, and the high-end smartphone: exploring the landscape of modern cross-device reaccess. In: Lyons, K., Hightower, J., Huang, E.M. (eds.) Pervasive 2011. LNCS, vol. 6696, pp. 1–18. Springer, Heidelberg (2011). Scholar
  5. 5.
    Luyten, K., Coninx, K.: Distributed user interface elements to support smart interaction spaces. In: Proceedings of the Seventh IEEE International Symposium on Multimedia, pp. 277–286. IEEE Computer Society (2005)Google Scholar
  6. 6.
    Sousa, J.P., Garlan, D.: Aura: an architectural framework for user mobility in ubiquitous computing environments. In: Proceedings of the IFIP 17th World Computer Congress - TC2 Stream/3rd IEEE/IFIP Conference on Software Architecture: System Design, Development and Maintenance. Kluwer, B.V. pp. 29–43 (2002)Google Scholar
  7. 7.
    Wäljas, M., Segerståhl, K., Väänänen-Vainio-Mattila, K., Oinas-Kukkonen, H.: Cross-platform service user experience: a field study and an initial framework. In: Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services, pp. 219–228. ACM, Lisbon (2010)Google Scholar
  8. 8.
    Jokela, T., Ojala, J., Olsson, T.: A diary study on combining multiple information devices in everyday activities and tasks. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 3903–3912. ACM, Seoul (2015)Google Scholar
  9. 9.
    Lucero, A., Holopainen, J., Jokela, T.: Pass-them-around: collaborative use of mobile phones for photo sharing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1787–1796. ACM, Vancouver (2011)Google Scholar
  10. 10.
    Nielsen, H.S., Olsen, M.P., Skov, M.B., Kjeldskov, J.: JuxtaPinch: exploring multi-device interaction in collocated photo sharing. In: Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices & Services. ACM (2014)Google Scholar
  11. 11.
    Ohta, T., Tanaka, J.: Pinch: an interface that relates applications on multiple touch-screen by ‘pinching’ gesture. In: Nijholt, A., Romão, T., Reidsma, D. (eds.) ACE 2012. LNCS, vol. 7624, pp. 320–335. Springer, Heidelberg (2012). Scholar
  12. 12.
    Hinckley, K., Ramos, G., Guimbretiere, F., Baudisch, P., Smith, M.: Stitching: pen gestures that span multiple displays. In: Proceedings of the Working Conference on Advanced Visual Interfaces. ACM (2004)Google Scholar
  13. 13.
    De Luca, A., Frauendienst, B.: A privacy-respectful input method for public terminals. In: Proceedings of the 5th Nordic Conference on Human-Computer Interaction: Building Bridges, pp. 455–458. ACM, Lund (2008)Google Scholar
  14. 14.
    Geel, M., Huguenin, D., Norrie, M.C.: PresiShare: opportunistic sharing and presentation of content using public displays and QR codes. In: Proceedings of the 2nd ACM International Symposium on Pervasive Displays. ACM (2013)Google Scholar
  15. 15.
    Chen, X.A., Grossman, T., Wigdor, D.J., Fitzmaurice, G.: Duet: exploring joint interactions on a smart phone and a smart watch. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 159–168. ACM, Toronto (2014)Google Scholar
  16. 16.
    Noh, W., Lee, M., Cheon, H., Kim, J., Lee, K., Cho, J.: TakeOut: drawing application using distributed user interface for being close to real experience. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, pp. 173–176. ACM, Heidelberg (2016)Google Scholar
  17. 17.
    Nguyen, N.T., Lee, H.: ‘Hop-to-select’ traverse with gestural input in an eye-off interaction. In: Proceedings of the 29th Australian Conference on Computer-Human Interaction, pp. 597–601. ACM, Brisbane (2017)Google Scholar
  18. 18.
    Horak, T., Badam, S.K., Elmqvist, N., Dachselt, R.: When David meets Goliath: combining smartwatches with a large vertical display for visual data exploration. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13. ACM, Montreal (2018)Google Scholar
  19. 19.
    Nguyen, N.T., Lee, H.: Understanding usability challenges in shifting between multiple devices during a task. In: 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, Kyoto (2019)Google Scholar
  20. 20.
    Shneiderman, B., Plaisant, C., Cohen, M., Jacobs, S., Elmqvist, N., Diakopoulos, N.: Designing the User Interface: Strategies for Effective Human-Computer Interaction. Pearson, London (2016)Google Scholar
  21. 21.
    Nielsen, J.: Usability Engineering. Morgan Kaufmann Publishers Inc, Burlington (1993)CrossRefGoogle Scholar
  22. 22.
    Gellersen, H., et al.: Supporting device discovery and spontaneous interaction with spatial references. Pers. Ubiquit. Comput. 13(4), 255–264 (2009)CrossRefGoogle Scholar
  23. 23.
    Nguyen, N.T., Lee, H.: SocioCon: a social circle for your interactive devices. In: Marcus, A., Wang, W. (eds.) DUXU 2018. LNCS, vol. 10919, pp. 623–639. Springer, Cham (2018). Scholar
  24. 24.
    Schilit, B., Adams, N., Want, R.: Context-aware computing applications. In: 1994 First Workshop on Mobile Computing Systems and Applications, WMCSA 1994. IEEE (1994)Google Scholar
  25. 25.
    Vogel, D., Balakrishnan, R.: Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In: Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology. ACM (2004)Google Scholar
  26. 26.
    Trimeche, M., et al.: Enhancing end-user experience in a multi-device ecosystem. In: Proceedings of the 4th International Conference on Mobile and Ubiquitous Multimedia, pp. 19–25. ACM, Christchurch (2005)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Singapore University of Technology and DesignSingaporeSingapore

Personalised recommendations