Mid-Air Interaction vs Smartphone Control for First-Person Navigation on Large Displays: A Comparative Study

  • Spyros VosinakisEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10851)


User navigation in 3D environments through public large-screen installations is mostly supported by mid-air interactions using motion sensors such as Microsoft Kinect. On the other hand, smartphones have been also used as external controllers of large-screen installations, and they might as well be effective in supporting 3D navigations. The aim of this study is to examine whether a smartphone-based control is a reliable alternative to mid-air interaction for 4-DOF fist-person navigation. We setup an experiment, where users had to navigate and complete a given task in two different scenes using three input modalities. The input modalities and interaction techniques were a known Kinect-based navigation method using body tilt and shoulder rotation, a smartphone-based control though tilting and rotating the device, and a traditional keyboard-based input used as a basis of comparison. We measured quantitative data through automated monitoring of users’ actions in the environment (time to finish the task, number of collisions, total collision time and distance travelled), as well as subjective user ratings through questionnaires. The results indicate that smartphone control performs at least as good as mid-air interactions with Kinect, and it was the preferred method of input in the majority of the users.


3D navigation Large screens Smartphone Kinect User study 


  1. 1.
    Fanini, B., D’Annibale, E., Demetrescu, E., Ferdani, D.: Engaging and Shared Gesture-based Interaction for Museums: The case study of K2R international Expo in Rome. In: Proceedings of Digital Heritage 2015, pp. 263–270. IEEE (2015)Google Scholar
  2. 2.
    Rufa, C., Pietroni, E., Pagano, A.: The Etruscanning project : Gesture-based interaction and user experience in the virtual reconstruction of the Regolini- Galassi tomb. In: Proceedings of 2013 Digital Heritage International Congress, pp. 653–660 (2013)Google Scholar
  3. 3.
    Castro, B.P., Velho, L., Kosminsky, D.: INTEGRARTE: digital art using body interaction. In: Proceedings of the Eighth Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (2012)Google Scholar
  4. 4.
    Häkkilä, J., Koskenranta, O., Posti, M., He, Y.: City landmark as an interactive installation: experiences with stone, water and public space. In: Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction, pp. 221–224 (2014)Google Scholar
  5. 5.
    Grammenos, D., Drossis, G., Zabulis, X.: Public Systems Supporting Noninstrumented Body-Based Interaction. In: Playful user Interfaces, pp. 25–45, Springer, Singapore (2014)Google Scholar
  6. 6.
    Krekhov, A., Emmerich, K., Babinski, M.: Gestures from the point of view of an audience : toward anticipatable interaction of presenters with 3D content. In: Proceedings of CHI 2017, pp. 5284–5294. ACM (2017)Google Scholar
  7. 7.
    Bergé, L.P., Perelman, G., Raynal, M., Sanza, C., Serrano, M., Houry-Panchetti, M., Cabanac, R., Dubois, E.: Smartphone-based 3D navigation technique for use in a museum exhibit. In: Proceedings of ACHI 2014, The Seventh International Conference on Advances in Computer-Human Interactions, pp. 252–257 (2014)Google Scholar
  8. 8.
    Cristie, V., Berger, M.: Game engines for urban exploration: bridging science narrative for broader participants. In: Nijholt, A. (ed.) Playable Cities, pp. 87–107, Springer, Singapore (2017). Scholar
  9. 9.
    Bowman, D.A., Koller, D., Hodges, L.F.: Travel in immersive virtual environments: an evaluation of viewpoint motion control techniques georgia institute of technology. In: Proceedings of the 1997 Virtual Reality Annual International Symposium, pp. 45–52. IEEE (1997)Google Scholar
  10. 10.
    Vosinakis, S., Xenakis, I.: A virtual world installation in an art exhibition: providing a shared interaction space for local and remote visitors. In: Proceedings of Re-thinking Technology in Museums (2011)Google Scholar
  11. 11.
    Cho, N., Shin, D., Lee, D., Kim, K., Park, J., Koo, M., Kim, J.: Intuitional 3D museum navigation system using kinect. In: Park, J., Barolli, L., Xhafa, F., Jeong, H.Y. (eds.) Information Technology Convergence. LNEE, vol. 253, pp. 587–596. Springer, Dordrecht (2013). Scholar
  12. 12.
    Baldauf, M., Adegeye, F., Alt, F., Harms, J.: Your browser is the controller: advanced web-based smartphone remote controls for public screens. In: Proceedings of the 5th ACM International Symposium on Pervasive Displays, pp. 175–181. ACM (2016)Google Scholar
  13. 13.
    Hornecker, E., Stifter, M.: Learning from interactive museum installations about interaction design for public settings. In: Proceedings of OzCHI 2006, pp. 135–142 (2006)Google Scholar
  14. 14.
    Kurdyukova, E., Obaid, M., André, E.: Direct, bodily or mobile interaction? comparing interaction techniques for personalized public displays. In: Proceedings of 11th International Conference on Mobile and Ubiquitous Multimedia (MUM 2012). ACM (2012)Google Scholar
  15. 15.
    Ardito, C., Buono, P., Costabile, M.F., Moro, A.: Interaction with large displays: a survey. ACM Comput. Surv. 47(3), 1–38 (2015)CrossRefGoogle Scholar
  16. 16.
    Jankowski, J., Hulin, T., Hachet, M.: A study of street-level navigation techniques in 3D digital cities on mobile touch devices. In: Proceedings of 3DUI 2014, IEEE Symposium on 3D User Interfaces, pp. 35–38, IEEE (2014)Google Scholar
  17. 17.
    Roupé, M., Bosch-Sijtsema, P., Johansson, M.: Interactive navigation interface for Virtual Reality using the human body. Comput. Environ. Urban Syst. 43, 42–50 (2014)CrossRefGoogle Scholar
  18. 18.
    Ren, G., Li, C., O’Neill, E., Willis, P.: 3D freehand gestural navigation for interactive public displays. IEEE Comput. Graph. Appl. 33(2), 47–55 (2013)CrossRefGoogle Scholar
  19. 19.
    Dias, P., Parracho, J., Cardoso, J., Ferreira, B.Q., Ferreira, C., Santos, B.S.: Developing and evaluating two gestural-based virtual environment navigation methods for large displays. In: Streitz, N., Markopoulos, P. (eds.) DAPI 2015. LNCS, vol. 9189, pp. 141–151. Springer, Cham (2015). Scholar
  20. 20.
    Guy, E., Punpongsanon, P., Iwai, D., Sato, K., Boubekeur, T.: LazyNav: 3D ground navigation with non-critical body parts. In: Proceedings of 3DUI 2015, IEEE Symposium on 3D User Interfaces, pp. 43–50. IEEE (2015)Google Scholar
  21. 21.
    Hernandez-Ibanez, L.A., Barneche-Naya, V., Mihura-Lopez, R.: A comparative study of walkthrough paradigms for virtual environments using kinect based natural interaction. In: Proceedings of the 22nd International Conference on Virtual System & Multimedia (VSMM), pp. 1–7 (2016)Google Scholar
  22. 22.
    Vajk, T., Coulton, P., Bamford, W., Edwards, R.: Using a mobile phone as a “Wii-like” controller for playing games on a large public display. Int. J. Comput. Games Technol. 2008(6), 6 (2008)CrossRefGoogle Scholar
  23. 23.
    Du, Y., Ren, H., Pan, G., Li, S.: Tilt & touch: mobile phone for 3D interaction. In: Proceedings of the 13th International Conference on Ubiquitous Computing, pp. 485–486. ACM (2011)Google Scholar
  24. 24.
    Shirazi, A. S., Winkler, C., Schmidt, A.: Flashlight interaction: a study on mobile phone interaction techniques with large displays. In: Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI 2009. ACM (2009)Google Scholar
  25. 25.
    Pazmino, P.J., Lyons, L.: An exploratory study of input modalities for mobile devices used with museum exhibits. In: Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems – CHI11, pp. 895–904. ACM (2011)Google Scholar
  26. 26.
    Joselli, M., Da Silva, J. R., Zamith, M., Clua, E., Pelegrino, M., Mendonça, E., Soluri, E.: An architecture for game interaction using mobile. In: Proceedings of the 4th International IEEE Consumer Electronic Society - Games Innovation Conference, IGiC 2012, pp. 1–5. IEEE (2012)Google Scholar
  27. 27.
    Liang, H.N., Trenchard, J., Semegen, M., Irani, P.: An exploration of interaction styles in mobile devices for navigating 3D environments. In: Proceedings of the 10th Asia Pacific conference on Computer human interaction - APCHI 2012, pp. 309–313. ACM (2012)Google Scholar
  28. 28.
    Liang, H., Shi, Y., Lu, F., Yang, J., Papangelis, K.: VRMController: an input device for navigation activities in virtual reality environments. In: Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry, pp. 455–460. ACM (2016)Google Scholar
  29. 29.
    Brondi, R., et al.: Evaluating the impact of highly immersive technologies and natural interaction on player engagement and flow experience in games. In: Chorianopoulos, K., Divitini, M., Hauge, J.B., Jaccheri, L., Malaka, R. (eds.) ICEC 2015. LNCS, vol. 9353, pp. 169–181. Springer, Cham (2015). Scholar
  30. 30.
    Burigat, S., Chittaro, L.: Navigation in 3D virtual environments: effects of user experience and location-pointing navigation aids. Int. J. Hum Comput Stud. 65(11), 945–958 (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Product and Systems Design EngineeringUniversity of the AegeanSyrosGreece

Personalised recommendations