Some Usability Issues of Augmented and Mixed Reality for e-Health Applications in the Medical Domain

  • Reinhold Behringer
  • Johannes Christian
  • Andreas Holzinger
  • Steve Wilkinson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4799)


Augmented and Mixed Reality technology provides to the medical field the possibility for seamless visualization of text-based physiological data and various graphical 3D data onto the patient’s body. This allows improvements in diagnosis and treatment of patients. For the patient, this technology offers benefits and further potential in therapy, rehabilitation and diagnosis, and explanation. Applications across the whole range of functions that affect the health sector from the physician, the medical student, to the patients are possible. However, the quality of the work of medical professionals is considerably influenced by both usefulness and usability of technology. Consequently, key issues in developing such applications are the tracking methodology, the display technology and most of all ensuring good usability. There have been several research groups who extended the state of the art in employing these technologies in the medical domain. However, only a few are addressing issues of Human-Computer Interaction, Interaction Design, and Usability Engineering. This paper provides a brief overview over the history and the most recent developments in this domain with a special focus on issues of user-centered development.


Human–Computer Interaction Augmented Reality Mixed Reality Visualization User-Centered Development e-Health Medicine 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bajura, M., Fuchs, H., Ohbuchi, R.: Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient. In: Proceedings of SIGGRAPH 1992. Computer Graphics, vol. 26, pp. 203–210 (1992)Google Scholar
  2. 2.
    State, A., McAllister, J., Neumann, U., Chen, H., Cullip, T., Chen, D.T., Fuchs, H.: Interactive Volume Visualization on a Heterogeneous Message-Passing Multicomputer. In: Proceedings of the 1995 ACM Symposium on Interactive 3D Graphics (Monterey, CA, April 9-12, 1995), Special issue of Computer Graphics, ACM SIGGRAPH, New York, pp. 69–74 (1995) Also UNC-CH Dept. of Computer Science technical report TR94-040, 1994Google Scholar
  3. 3.
    State, A., Chen, D.T., Tector, C., Brandt, A., Chen, H., Ohbuchi, R., Bajura, M., Fuchs, H.: Case Study: Observing a Volume-Rendered Fetus within a Pregnant Patient. In: Proceedings of IEEE Visualization 1994, pp. 364–368. IEEE Computer Society Press, Los Alamitos (1994)CrossRefGoogle Scholar
  4. 4.
    State, A., Livingston, M.A., Hirota, G., Garrett, W.F., Whitton, M.C., Fuchs, H., Pisano, E.D.: Technologies for Augmented-Reality Systems: realizing Ultrasound-Guided Needle Biopsies. In: ACM SIGGRAPH. Proceedings of SIGGRAPH 1996 (New Orleans, LA, August 4-9, 1996). In Computer Graphics Proceedings, Annual Conference Series, pp. 439–446 (1996)Google Scholar
  5. 5.
    State, A., Keller, K., Fuchs, H.: Simulation-Based Design and Rapid Prototyping for a Pallax-Free, Orthoscopic Video See-Through Head-Mounted Display. In: Proc. ISMAR 2005, pp. 28–31 (2005)Google Scholar
  6. 6.
    Fuchs, H., Livingston, M.A., Raskar, R., Colucci, D., Keller, K., State, A., Crawford, J.R., Rademacher, P., Drake, S.H., Anthony, A., Meyer, M.D.: Augmented Reality Visualization for Laparoscopic Surgery. In: Wells, W.M., Colchester, A.C.F., Delp, S.L. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 11–13. Springer, Heidelberg (1998)Google Scholar
  7. 7.
    Stockmans, F.: Preoperative 3D virtual planning and surgery for the treatment of severe madelung’s deformity. In: Proc of Societe Francaise Chirurgie de la main, XLI Congres, vol. CP3006, pp. 314–317 (2005)Google Scholar
  8. 8.
    State, A., Keller, K., Rosenthal, M., Yang, H., Ackerman, J., Fuchs, H.: Stereo Imagery from the UNC Augmented Reality System for Breast Biopsy Guidance. In: Proc. MMVR 2003 (2003)Google Scholar
  9. 9.
    Southern, S.J., Shamsian, N., Wilkinson, S.: Real hand© - A 3-dimensional interactive web-based hand model - what is the role in patient education? In: XLIe Congrès national de la Société Française de Fhirurgie de la Fain, Paris (France), (15-17December, 2005)Google Scholar
  10. 10.
    Shuhaiber, J.H.: Augmented Reality in Surgery. ARCH SURG 139, 170–174 (2004)CrossRefGoogle Scholar
  11. 11.
    Grimson, W.E.L., Lozano-Perez, T., Wells III, W.M., Ettinger, G.J., White, S.J., Kikinis, R.: An Automatic Registration Method for Frameless Stereotaxy, Image Guided Surgery, and Enhanced Reality Visualization. In: Computer Vision and Pattern Recognition Conference, Seattle (June 1994)Google Scholar
  12. 12.
    Morgan, F.: Developing a New Medical Augmented Reality System. Tech. report CMU-RI-TR-96-19, Robotics Institute, Carnegie Mellon University, (May 1996)Google Scholar
  13. 13.
  14. 14.
    Schwald, B., Seibert, H., Weller, T.: A Flexible Tracking Concept Applied to Medical Scenarios Using an AR Window. In: ISMAR 2002 (2002)Google Scholar
  15. 15.
    Schnaider, M., Schwald, B.: Augmented Reality in Medicine – A view to the patient’s inside. Computer Graphic topics, Issue 1, INI-GraphicsNet Foundation, Darmstadt (2004)Google Scholar
  16. 16.
    International Workshop on Medical Imaging and Augmented Reality (MIAR),
  17. 17.
    Fischer, J., Bartz, D., Straßer, W.: Occlusion Handling for Medical Augmented Reality. In: VRST. Proceedings of the ACM symposium on Virtual reality software and technology, Hong Kong (2004)Google Scholar
  18. 18.
    Sielhorst, T., Obst, T., Burgkart, R., Riener, R., Navab, N.: An Augmented Reality Delivery Simulator for Medical Training. In: International Workshop on Augmented Environments for Medical Imaging - MICCAI Satellite Workshop (2004)Google Scholar
  19. 19.
    Birkfellner, W.M., Figl, K., Huber, F., Watzinger, F., Wanschitz, R., Hanel, A., Wagner, D., Rafolt, R., Ewers, H., Bergmann, H.: The Varioscope AR – A head-mounted operating microscope for Augmented Reality. In: Proc. of the 3rd International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 869–877 (2000)Google Scholar
  20. 20.
    Reitinger, B., Werlberger, P., Bornik, A., Beichel, R., Schmalstieg, D.: Spatial Measurements for Mecial Augmented Reality. In: ISMAR 2005. Proc. of the 4th IEE and ACM International Symposium on Mixed and Augmented Reality, pp. 208–209 (October 2005)Google Scholar
  21. 21.
    Riva, G., Morganti, F., Villamira, M.: Immersive Virtual Telepresence: Virtual Reality meets eHealth. In: Cybertherapy-Internet and Virtual Reality as Assessment and Rehabilitation Tools for Clinical Psychology and Neuroscience, IOS Press, Amsterdam (2006)Google Scholar
  22. 22.
    Nischelwitzer, A., Lenz, F.J., Searle, G., Holzinger, A.: Some Aspects of the Development of Low-Cost Augmented Reality Learning Environments as Examples for Future Interfaces in Technology Enhanced Learning. In: Universal Access to Applications and Services. LNCS, vol. 4556, pp. 728–737. Springer, New YorkGoogle Scholar
  23. 23.
    Juan, M.C., Alcaniz, M., Monserrat, C., Botella, C., Banos, R.M., Guerrero, B.: Using Augmented Reality to Treat Phobias. IEEE Comput. Graph. Appl. 25, 31–37 (2005)CrossRefGoogle Scholar
  24. 24.
  25. 25.
    Gaggioli, A., Morganti, F., Meneghini, A., Alcaniz, M., Lozano, J.A., Montesa, J., Martínez, J.M., Walker, R., Lorusso, I., Riva, G.: Abstracts from CyberTherapy 2005 The Virtual Mirror: Mental Practice with Augmented Reality for Post-Stroke Rehabilitation. CyberPsychology & Behavior 8(4) (2005)Google Scholar
  26. 26.
    Bimber, O., Raskar, R.: Spatial Augmented Reality: Merging Real and Virtual Worlds. A. K. Peters, Ltd (2005)Google Scholar
  27. 27.
    Maurer, H., Oliver, R.: The future of PCs and implications on society. Journal of Universal Computer Science 9(4), 300–308 (2003)Google Scholar
  28. 28.
  29. 29.
  30. 30.
    del Valle, A., Opalach, A.: The Persuasive Mirror: Computerized Persuasion for Healthy Living. In: Proceedings of Human Computer Interaction International, HCI International, Las Vegas, US (July 2005)Google Scholar
  31. 31.
    Nestler, S., Dollinger, A., Echtler, F., Huber, M., Klinker, G.: Design and Development of Virtual Patients. In: 4th Workshop VR/AR, Weimar (15 July, 2007)Google Scholar
  32. 32.
    Elkin, P.L., Sorensen, B., De Palo, D., Poland, G., Bailey, K.R., Wood, D.L., LaRusso, N.F.: Optimization of a research web environment for academig internal medicine faculty. Journal of the American Medical Informatics Association 9(5), 472–478 (2002)CrossRefGoogle Scholar
  33. 33.
    Hesse, B.W., Shneiderman, B.: eHealth research from the user’s perspective. American Journal of Preventive Medicine 32(5), 97–103 (2007)CrossRefGoogle Scholar
  34. 34.
    Rhodes, M.L.: Computer Graphics and Medicine: A Complex Partnership IEEE. Computer Graphics and Applications 17(1), 22–28 (1997)CrossRefGoogle Scholar
  35. 35.
    Holzinger, A., Geierhofer, R., Ackerl, S., Searle, G.: CARDIAC@VIEW: The User Centered Development of a new Medical Image Viewer. In: Zara, J.S.J. (ed.) Central European Multimedia and Virtual Reality Conference (available in Eurographics Library), pp. 63–68, Czech Technical University (2005)Google Scholar
  36. 36.
    Holzinger, A.: Usability Engineering for Software Developers. Communications of the ACM 48(1), 71–74 (2005)CrossRefGoogle Scholar
  37. 37.
    Gould, J.D., Lewis, C.: Designing for usability: key principles and what designers think. Communications of the ACM 28(3), 300–331 (1985)CrossRefGoogle Scholar
  38. 38.
    Seffah, A., Metzker, E.: The obstacles and myths of usability and software engineering. Communications of the ACM 47(12), 71–76 (2004)CrossRefGoogle Scholar
  39. 39.
    Sutcliffe, A., Gault, B., Shin, J.E.: Presence, memory and interaction in virtual environments. International Journal of Human-Computer Studies 62(3), 307–332 (2005)CrossRefGoogle Scholar
  40. 40.
    Wilson, J.R., D’Cruz, M.: Virtual and interactive environments for work of the future. International Journal of Human-Computer Studies 64(3), 158–169 (2006)Google Scholar
  41. 41.
    Dünser, A., Grassert, R., Seichter, H., Billinghurst, M.: Applying HCI principles to AR systems design. In: MRUI 2007. 2nd International Workshop at the IEEE Virtual Reality 2007 Conference, Charlotte, North Carolina, USA (March 11, 2007)Google Scholar
  42. 42.
    Billinghurst, M.: Designing for the masses. INTERFACE HITLabNZ, Issue 14 (July2007)Google Scholar
  43. 43.
    Gabbard, D.H., Swan, J.E.: User-centered design and evaluation of virtual environments. IEEE Computer Graphics and Applications 19(6), 51–59 (1999)CrossRefGoogle Scholar
  44. 44.
    Freeman, S.E.A., Pearson, D.E., Ijsselsteijn, W.A.: Effects of sensory information and prior experience on direct subjective ratings of presence. Presence Teleoperators and Virtual Environments 8, 1–13 (1999)CrossRefGoogle Scholar
  45. 45.
    Kalawsky,: VRUSE A computerised diagnostic tool for usability evaluation of virtual/synthetic environment systems. Applied Ergonomics 30(1), 11–25 (1999)CrossRefGoogle Scholar
  46. 46.
    Bowman, D., Hodges, L.: An Evaluation of Techniques for Grabbing and Manipulating Remote Objects in Immersive Virtual Environment. In: Proceedings of the ACM Symposium on Interactive 3D Graphics, ACM Press, New York (1997)Google Scholar
  47. 47.
    Slater, M., Linakis, V., Usoh, M., Kooper, R.: Immersion, Presence and Performance in Virtual Environments: An Experiment Using Tri-Dimensional Chess. Available online at staff/m.slater/Papers/Chess/index.html (1996)Google Scholar
  48. 48.
    Lombard, M., Ditton, T.: At the heart of it all: the concept of presence. Journal of Computer Mediated Communication 3(2),
  49. 49.
    Blach, R., Simon, A., Riedel.: Experiences with user interactions in a CAVE-like projection environment. In: Proceedings Seventh International Conference on Human-Computer Interaction (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Reinhold Behringer
    • 1
  • Johannes Christian
    • 1
  • Andreas Holzinger
    • 2
  • Steve Wilkinson
    • 1
  1. 1.Leeds Metropolitan University, Leeds, LS6 3QS, United Kingdom, Innovation North – Faculty for Information and Technology 
  2. 2.Medical University Graz, 8036 Graz, Austria, Institute for Medical Informatics, Statistics & Documentation (IMI), Research Unit HCI4MED 

Personalised recommendations