A Multimodal Perceptual User Interface for Collaborative Environments

  • Giancarlo Iannizzotto
  • Francesco La Rosa
  • Carlo Costanzo
  • Pietro Lanzafame
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3617)


In this paper a 3D graphics-based remote collaborative environment is introduced. This system is able to provide multiclient and multimedia communication, and exploits a novel multimodal user interaction paradigm based on hand gesture and perceptual user interfaces. The use of machine vision technologies and a user-centered approach produce a highly usable and natural human-computer interface, allowing even untrained users a realistic and relaxing experience for long and demanding tasks. We then note and motivate that such an application can be considered as an Augmented Reality application; according to this view, we describe our platform in terms of long-term usability and comfort of use. The proposed system is expected to be useful in remote interaction with dynamic environments. To illustrate our work, we introduce a proof-of-concept multimodal, bare-hand application and discuss its implementation and the obtained experimental results.


Augmented Reality Hand Gesture Multimedia Communication Collaborative Environment Video Projector 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Leung, W.H., Goudeaux, K., Panichpapiboon, S., Wang, S.B., Chen, T.: Networked intelligent collaborative environment (netice). In: Proc. of the IEEE Intl. Conf. on Multimedia and Expo., New York (2000)Google Scholar
  2. 2.
    Rich, C., Waters, R.C., Strohecker, C., Schabes, Y.T., Freeman, W., Torrance, M.C., Golding, A.R., Roth, M.: Demonstration of an interactive multimedia environment. IEEE Computer 27 (1994)Google Scholar
  3. 3.
    Fersha, A.: Distributed interaction in virtual spaces. In: Proc. Of the 3rd International WS on Distributed Interactive Simulation and Real Time Applications. IEEE CS-Press, Los Alamitos (1999)Google Scholar
  4. 4.
    Carlsson, C., Hagsand, O.: Dive - a platform for multi-user virtual environments. IEEE Computers and Graphics 17(6) (1993)Google Scholar
  5. 5.
    (Smartverse web page),
  6. 6.
    Ferscha, A.: Workspace awareness in mobile virtual teams. In: WETICE 2000. IEEE Computer Society, Los Alamitos (2000)Google Scholar
  7. 7.
    Rekimoto, J.: Gesturewrist and gesturepad: Unobtrusive wearable interaction devices. In: Proceedings of ISWC 2001, Zurich (2001)Google Scholar
  8. 8.
    Towles, H., Chen, W.C., Yang, R., Kum, S.U., Fuchs, H., Kelshikar, N., Mulligan, J., Daniilidis, K., Holden, L., Zeleznik, B., Sadagic, A., Lanier, J.: 3d tele-collaboration over internet2. In: Proceedings of International Workshop on Immersive Telepresence (ITP 2002), Juan Les Pins, France (2002)Google Scholar
  9. 9.
    Ye, G., Corso, J., Burschka, D., Hager, D.: Vics: A modular vision-based hci framework. In: Proceedings of ICVS 2003, Graz, Austria (2003)Google Scholar
  10. 10.
    Azuma, R.: A survey of augmented reality. Presence: Teleoperators and Virtual Environments 6, 355–385 (1997)Google Scholar
  11. 11.
    Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances in augmented reality. IEEE Computer Graphics and Applications 21, 34–47 (2001), CrossRefGoogle Scholar
  12. 12.
    Costanzo, C., Iannizzotto, G., LaRosa, F.: Virtualboard: Real-time visual gesture recognition for natural human-computer interaction. In: Proc. of the IEEE IPDPS 2003, Nice, France (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Giancarlo Iannizzotto
    • 1
  • Francesco La Rosa
    • 1
  • Carlo Costanzo
    • 1
  • Pietro Lanzafame
    • 1
  1. 1.VisiLab, Faculty of EngineeringUniversity of Messina 

Personalised recommendations