Virtual Reality and Augmented Reality

  • Yasser El Miedany


Medical education is transforming thanks to medical schools adopting innovations to prepare new clinicians, paramedics as well as nurses for emerging models of care. In today’s rapidly changing healthcare system, physicians and care teams require new capabilities to best practice medicine and serve their patients. They must understand value-based payment models, how to operate in a patient-centred system – rather than one centred on the physician – and learn to incorporate new technologies into the practice of medicine. Virtual reality/augmented reality and virtual patients technologies have the potential to make interactive, customizable 3D virtual patients effective for educating healthcare professionals with vast applications. Also, they are valuable for undergraduate basic as well as clinical sciences learning. There are still some challenges of developing intelligent machines that appear to think and behave like humans. This chapter will discuss the science behind virtual reality technology, the virtual reality environments for health professional education and the use of virtual reality/augmented reality to improve the clinicians’ skills. The chapter will expand to discuss designing of realistic virtual patients, advantages and disadvantages of VR and AR for virtual patients as well as virtual reality/augmented reality in medical education. The chapter will conclude with discussion of the use of virtual reality/augmented reality for patient management.


Virtual patients Virtual reality Augmented reality Natural interface Natural language processing (NLP) Virtual reality environment Natural user interface 


  1. 1.
    Strangman N, Hall T. Virtual reality/simulations. Wakefield: National Center on Accessing the General Curriculum; 2003.Google Scholar
  2. 2.
    Mantovani F, Castelnuovo G, Gaggioli A, Riva G. Virtual reality training for health-care professionals. Cyberpsychol Behav. 2003;6(4):389–95.CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Rasmussen K, Belisario JM, Wark PA, Molina JA, Loong SL, Cotic Z, et al. Offline eLearning for undergraduates in health professions: a systematic review of the impact on knowledge, skills, attitudes and satisfaction. J Global Health. 2014;4(1):010405.CrossRefGoogle Scholar
  4. 4.
    Bowman DA, McMahan RP. Virtual reality: how much immersion is enough? Computer. 2007;40(7):36–43.CrossRefGoogle Scholar
  5. 5.
    De Luca R, Lo Buono V, Leo A, Russo M, Aragona B, Leonardi S, et al. Use of virtual reality in improving post-stroke neglect: promising neuropsychological and neurophysiological findings from a case study. Appl Neuropsychol Adult. 2017;1:1–5. Scholar
  6. 6.
    Greenleaf W. Medical Applications of Virtual Reality. Accessed on 27 May 2018.
  7. 7.
    Pourazar M, Mirakhori F, Hemayattalab R, Bagherzadeh F. Use of virtual reality intervention to improve reaction time in children with cerebral palsy: a randomized controlled trial. Dev Neurorehabil. 2017;1:1–6. Scholar
  8. 8.
    Van Bennekom MJ, De Koning PP, Denys D. Virtual reality objectifies the diagnosis of psychiatric disorders: a literature review. Front Psychol. 2017;8:163. Scholar
  9. 9.
    Augmented Reality vs. Virtual Reality – what’s the difference?
  10. 10.
  11. 11.
    Milgram P, Kishino F. A taxonomy of mixed reality visual displays. In: IEICE TRANSACTIONS on information and systems. 1994. p. 1321–9.Google Scholar
  12. 12.
    Dalgarno B, Lee MJ. What are the learning affordances of 3-D virtual environments? Br J Educ Technol. 2010;41(1):10–32.CrossRefGoogle Scholar
  13. 13.
    Psotka J. Immersive training systems: virtual reality and education and training. Instr Sci. 1995;23(5–6):405–31.CrossRefGoogle Scholar
  14. 14.
    Schultheis MT, Rizzo AA. The application of virtual reality technology in rehabilitation. Rehabil Psychol. 2001;46(3):296.CrossRefGoogle Scholar
  15. 15.
    Hansen MM. Versatile, immersive, creative and dynamic virtual 3-D healthcare learning environments: a review of the literature. J Med Internet Res. 2008;10(3):e26.CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Saxena N, Kyaw BM, Vseteckova J, Dev P, Paul P, Lim KTK, Kononowicz A, Masiello I, Tudor Car L, Nikolaou CK, Zary N, Car J. Virtual reality environments for health professional education. Cochrane Database Syst Rev. 2016;(2):CD012090.
  17. 17.
    Jarmon L, Traphagan T, Mayrath M, Trivedi A. Virtual world teaching, experiential learning, and assessment: an interdisciplinary communication course in second life. Comput Educ. 2009;53(1):169–82.CrossRefGoogle Scholar
  18. 18.
    Kurtz S, Silverman J, Draper J. Teaching and learning communication skills in medicine. Oxford: CRC Press; 1998.Google Scholar
  19. 19.
    Fertleman C, Aubugeau-Williams P, Sher C, Lim A-N, Lumley S, Delacroix S, Pan X. A discussion of virtual reality as a new tool for training healthcare professionals. Front Public Health. 2018;6:44. Scholar
  20. 20.
    Pan X, Slater M, Beacco A, Navarro X, Bellido Rivas AI, Swapp D, et al. The responses of medical general practitioners to unreasonable patient demand for antibiotics––a study of medical ethics using immersive virtual reality. PLoS One. 2016;11(2):e0146837. Scholar
  21. 21.
    Zielke MA, Zakhidov D, Hardee G, Evans L, Lenox S, Orr N, Fino D, Mathialagan G. Developing virtual patients with VR/AR for a natural user interface in medical teaching.
  22. 22.
    Falcao C, Lemos AC, Soares M. Evaluation of natural user interface: a usability study based on the leap motion device. Procedia Manuf. 2015;3:5490–5.CrossRefGoogle Scholar
  23. 23.
    Mavromihelaki E, et al. Cyberball3D+: A 3D serious game for fMRI investigating social exclusion and empathy. In: 2014 6th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES), Valletta. 2014. p. 1–8.Google Scholar
  24. 24.
    Kleinsmith A, Rivera-Gutierrez D, Finney G, Cendan J, Lok B. Understanding empathy training with virtual patients. Comput Hum Behav. 2015;52:151–8. [Online].CrossRefGoogle Scholar
  25. 25.
    Raffii A, Zuccarino T. Method and system enabling natural user interface gestures with user wearable glasses. U.S. Patent 9310891. 2016.Google Scholar
  26. 26.
    “Natural User Interface”. In: NUI Group. 2011. [Online]. Available: Accessed 28 May 2018.
  27. 27.
    Turing A. I.—Computing machinery and intelligence. Mind. 1950;(236):433–60.CrossRefGoogle Scholar
  28. 28.
    Talbot T, Sagae K, John B, Rizzo A. Sorting out the virtual patient. Int J Gaming Comput Mediat Simul. 2012;4(3):1–19.CrossRefGoogle Scholar
  29. 29.
    “SimCoach.” 2013. [Online]. Available: Accessed 26 May 2018.
  30. 30.
    Donald CC, Sokolowski JA, Banks CM, editors. The digital patient: advancing healthcare, research, and education. Hoboken: Wiley; 2015.Google Scholar
  31. 31.
    De Vault D, et al. “SimSensei kiosk,” International Foundation for Autonomous Agents and Multiagent Systems. 2014. p. 1061–8. [Online]. Available:
  32. 32.
    Virtual human Toolkit. 2017. [Online]. Available: Accessed May 2018.
  33. 33.
    Miller D. Intelligent assistant landscape shows slow growth but huge potential. VentureBeat. 2016. [Online]. Available: Accessed 28 May 2018.
  34. 34.
    What is Cortana? Microsoft support. 2016. [Online]. Available: Accessed 28 May 2018.
  35. 35.
    Manuvinakurike R, Paetzel M, Qu C, Schlangen D, De Vault D. Toward incremental dialogue act segmentation in fast-paced interactive dialogue systems. In: 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 2016.Google Scholar
  36. 36.
    Sanchez-Vives MV, Slater M. From presence to consciousness through virtual reality: abstract: nature reviews neuroscience. Nat Rev Neurosci. 2005;6(4):332–9. [Online]. Available: Scholar
  37. 37.
    Johnson DM. Introduction to and review of simulator sickness research. Defense Technical Information Center; 2005.Google Scholar
  38. 38.
    Oculus VR. LLC, Simulator sickness. 2016. [Online]. Available: Accessed 28 May 2018.
  39. 39.
    Hendrix C, Barfield W. The sense of presence within auditory virtual environments. Presence Teleop Virt. 1996;5(3):290–301. [Online]. Available: Scholar
  40. 40.
    Pellegrini RS. Quality assessment of auditory virtual environments. Georgia Institute of Technology; 2001. [Online]. Available:
  41. 41.
    Oculus VR. LLC, 3D Audio spatialization. 2016. [Online]. Available: Accessed 28 May 2018.
  42. 42.
    Lalwani M. 3D audio is the secret to HoloLens’ convincing holograms. 2016. [Online]. Available: Accessed 28 May 2018.
  43. 43.
    Bright P. Microsoft sheds some light on its mysterious holographic processing unit. Condé Nast. 2016. [Online]. Available: Accessed 28 May 2018.
  44. 44.
    Xiao R, Benko H. Augmenting the field-of-view of head-mounted displays with sparse peripheral displays. In: The 2016 CHI Conference. 2016.Google Scholar
  45. 45.
    Exporting and building a unity visual studio solution. [Online]. Available: Accessed 28 May 2018.
  46. 46.
    Dumay A, Jense G. Endoscopic surgery simulation in a virtual environment. Comput Biol Med (United States). 1995;25(2):139–48.CrossRefGoogle Scholar
  47. 47.
    Knuth TE, Wilson A, Oswald SG. Military training at civilian trauma centers: the first year’s experience with the regional trauma network. Mil Med (United States). 1998;163(9):608–41.CrossRefGoogle Scholar
  48. 48.
    Hoffman H, Vu D. Virtual reality: teaching tool of the twenty-first century. Acad Med (United States). 1997;72(12):1076–81.CrossRefGoogle Scholar
  49. 49.
    Jerant AF, Epperly TD, Marionneaux RD. Medical support for operation cooperative nugget ‘95’: joint readiness training in the post-cold war era. Mil Med (United States). 1997;162(11):702–5.Google Scholar
  50. 50.
    Pickett R. Joint readiness training center integration of live and constructive forces. I/ITSEC (CD-ROM). 1998.Google Scholar
  51. 51.
    McCarthy J, Johnston J, Paris C. Toward development of a tactical decision-making under stress integrated trainer. I/ITSEC (CD-ROM). 1998.Google Scholar
  52. 52.
    Arbor A. An immersive virtual reality platform for medical education: introduction to the medical readiness trainer. In: Proceedings of the 33rd Hawaii International Conference on System Sciences. 2000.Google Scholar
  53. 53.
    Takabayashi K, Fujikawa K, Suzuki T, Yamazakia S, Honda M, Amaral M, Satomura Y, Yoshida S, Tomioka H. Implementation and evaluation of computerized patient management problems. Medinfo (Canada). 1995;8(PT 2):1218–21.Google Scholar
  54. 54.
    Kraft S, Honebein P, Prince M, Marrero D. The SOCRATES curriculum: an innovative integration of technology and theory in medical education. J Audivo Media Med (England). 1997;20(4):166–71.CrossRefGoogle Scholar
  55. 55.
    Kreines M, Udal’tsovlu A, Pogromov A, Ershov V, Active AS. Computer systems for medical education: approach and implementation. Vestn Ross Akad Med Nauk (Russia). 1995;10:68–71.Google Scholar
  56. 56.
    Kaltenborn KF, Rienhoff O. Virtual reality in medicine. Methods Inf Med (Germany). 1992;32(5):407–17.Google Scholar
  57. 57.
    Larsson JE, Hayes-Roth B, Gaba DM, Smith BE. Evaluation of a medical diagnosis system using simulator test scenarios. Artif Intell Med (Netherlands). 1997;11(2):119–40.CrossRefGoogle Scholar
  58. 58.
    Fairén M, Farrés M, Moyés J, Insa E. Virtual reality to teach anatomy. Eurographics. 2017. Bourdin JJ, Shesh A, editors. 2017;
  59. 59.
    Dickey M. Brave new (interactive) worlds: a review of the design affordances and constrains of two 3d virtual worlds as interactive learning environments. Interact Learn Environ. 2005;13(1–2):121–37.CrossRefGoogle Scholar
  60. 60.
    Kleinginna PR, Kleinginna AM. A categorized list of emotion definitions, with suggestions for a consensual definition. Motiv Emot. 1981;5(4):345–79.CrossRefGoogle Scholar
  61. 61.
    Boctor L. Active-learning strategies: the use of a game to reinforce learning in nursing education. A case study. Nurse Educ Pract. 2013;13(2):96–100.CrossRefPubMedGoogle Scholar
  62. 62.
    Huang H-M, Rauch U, Liaw SS. Investigating learners’ attitudes toward virtual reality learning environments: based on a constructivist approach. Comput Educ. 2010;55(3):1171–82.CrossRefGoogle Scholar
  63. 63.
    Nicholson DT, Chalk C, Funnell WRJ, Daniel SJ. Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model. Med Educ. 2006;40(11):1081–7.CrossRefPubMedGoogle Scholar
  64. 64.
    Codd A, Choudhury B. Virtual reality anatomy: is it comparable with traditional methods in the teaching of human forearm musculoskeletal anatomy? Anat Sci Educ. 2011;4(3):119–25.CrossRefPubMedGoogle Scholar
  65. 65.
    Pan Z, Cheok AD, Yang H, Zhu J, Shi J. Virtual reality and mixed reality for virtual learning environments. Comput Graph. 2006;30(1):20–8.CrossRefGoogle Scholar
  66. 66.
    Vaughan N, Dubey VN, Wainwright TW, Middleton RG. A review of virtual reality based training simulators for orthopaedic surgery. Med Eng Phys. 2016;38:59–71.CrossRefPubMedGoogle Scholar
  67. 67.
    Aziz HA. Virtual reality programs applications in healthcare. J Health Med Informat. 2018;9:305. Scholar
  68. 68.
    Anderson PL, Price M, Edwards SM, Obasaju MA, Schmertz SK, et al. Virtual reality exposure therapy for social anxiety disorder: a randomized controlled trial. J Consult Clin Psychol. 2013;81:751–60.CrossRefPubMedGoogle Scholar
  69. 69.
    Parsons TD, Rizzo AA. Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: a meta-analysis. J Behav Ther Exp Psychiatry. 2008;39:250–61.CrossRefPubMedGoogle Scholar
  70. 70.
    Weiderhold MD, Weiderhold BK. Virtual reality and interactive simulation for pain distraction. Pain Med. 2007;1:182–8.CrossRefGoogle Scholar
  71. 71.
  72. 72.
    Aim F, Lonjon G, Hannouche D, Nizard R. Effectiveness of virtual reality training in orthopaedic surgery. Arthroscopy. 2016;32:224–32.CrossRefPubMedPubMedCentralGoogle Scholar
  73. 73.
    Valmaggia LR, Latif L, Kempton MJ, Ruscalafell M. Virtual reality in the psychological treatment for mental health problems: an systematic review of recent evidence. Psychiatry Res. 2016;236:189–95.CrossRefPubMedPubMedCentralGoogle Scholar
  74. 74.
    Wiederhold MD, Gao K, Wiederhold BK. Clinical use of virtual reality distraction system to reduce anxiety and pain in dental procedures. Cyberpsychol Behav Soc Netw. 2014;17:359–65.CrossRefPubMedPubMedCentralGoogle Scholar
  75. 75.
    Rutter CE, Dahhlquist LM, Weiss KE. Sustained efficacy of virtual reality distraction. J Pain. 2008;10:391–7.CrossRefGoogle Scholar
  76. 76.
    Schneider SM, Hood LE. Virtual reality: a distraction intervention for chemotherapy. Oncol Nurs Forum. 2007;34:39–46.CrossRefPubMedPubMedCentralGoogle Scholar
  77. 77.
    Broeren J, Bjorkdahl A. Virtual rehabilitation after stroke. Stud Health Technol Inform. 2008;136:77–82.PubMedPubMedCentralGoogle Scholar
  78. 78.
    Girard B, Turcotte V. Crushing virtual cigarettes reduces tobacco addiction and treatment discontinuation. Cyberpsychol Behav. 2009;12:477–83.CrossRefGoogle Scholar
  79. 79.
    Shin HH, Kim KM. Virtual reality for cognitive rehabilitation after brain injury: a systematic review. J Phys Ther Sci. 2015;27:2999–3002.CrossRefPubMedPubMedCentralGoogle Scholar
  80. 80.
    Hanna MG, Ahmed I, Nine J, Prajapati S, Pantanowitz L, et al. Virtual reality training for healthcare professionals. Cyberpsychol Behav. 2003;6:389–94.CrossRefGoogle Scholar
  81. 81.
    Li A, Montano Z, Chen VJ, Gold JI. Virtual reality and pain management: current trends and future directions. Pain Manag. 2011;1:147–57.CrossRefPubMedPubMedCentralGoogle Scholar
  82. 82.
    Mejiden OA, Schijven MP. The value of haptic feedback in conventional and robotic assisted minimal invasive surgery and virtual reality training: a current review. Surg Endosc. 2009;23:1180–90.CrossRefGoogle Scholar
  83. 83.
    Ojha U, Mohammed R, Vivekanantham S. Should there be greater exposure to interventional radiology in the undergraduate curriculum? Adv Med Educ Pract. 2017;8:791–5.CrossRefPubMedPubMedCentralGoogle Scholar
  84. 84.
    Pourmand A, Davis S, Lee D, Barber S, Sikka N. Emerging utility of virtual reality as a multidisciplinary tool in clinical medicine. Games Health J. 2017;6:263–70.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yasser El Miedany
    • 1
    • 2
  1. 1.King’s College London, Darent Valley HospitalDartfordUK
  2. 2.Rheumatology and RehabilitationAin Shams UniversityCairoEgypt

Personalised recommendations