Extended Reality in Medical Practice

  • Christopher Andrews
  • Michael K. Southworth
  • Jennifer N. A. SilvaEmail author
  • Jonathan R. SilvaEmail author
State-of-the-Art Informatics (M Majmudar, Section Editor)
Part of the following topical collections:
  1. Topical Collection on State-of-the-Art Informatics


Purpose of review

Advances in display technology and computing have led to new devices capable of overlaying digital information onto the physical world or incorporating aspects of the physical world into virtual scenes. These combinations of digital and physical environments are referred to as extended realities. Extended reality (XR) devices offer many advantages for medical applications including realistic 3D visualization and touch-free interfaces that can be used in sterile environments. This review introduces extended reality and describes how it can be applied to medical practice.

Recent findings

The 3D displays of extended reality devices are valuable in situations where spatial information such as patient anatomy and medical instrument position is important. Applications that take advantage of these 3D capabilities include teaching and pre-operative planning. The utility of extended reality during interventional procedures has been demonstrated with through 3D visualizations of patient anatomy, scar visualization, and real-time catheter tracking with touch-free software control.


Extended reality devices have been applied to education, pre-procedural planning, and cardiac interventions. These devices excel in settings where traditional devices are difficult to use, such as in the cardiac catheterization lab. New applications of extended reality in cardiology will continue to emerge as the technology improves.


Extended reality devices Informatics Display technology 


Compliance with Ethical Standards

Conflict of Interest

Christopher Andrews, Michael K. Southworth, Jennifer N. A. Silva, and Jonathan R. Silva each report grants from Children’s Discovery Institute and National Institute of Health.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

References and Recommended Reading

Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance

  1. 1.
    The Stanford virtual heart – revolutionizing education on congenital heart defects [Internet]. Available from:
  2. 2.
    Bruckheimer E, Rotschild C, Dagan T, Amir G, Kaufman A, Gelman S, et al. Computer-generated real-time digital holography: first time use in clinical medical imaging. Eur Heart J Cardiovasc Imaging. 2016;17:845–9.CrossRefGoogle Scholar
  3. 3.
    • Silva JN, Southworth MK, Dalal A, Van Hare GF. Improving visualization and interaction during transcatheter ablation using an augmented reality system: first-in-human experience (abstr). Circulation. 2017;136 Our prototype electrophysiology mapping software developed for the Microsoft HoloLens demonstrates the power of XR by visualizing patient-specific 3D anatomy and real-time catheter locations with a completely sterile user interface.Google Scholar
  4. 4.
    Loy Rodas N, Barrera F, Padoy N. See it with your own eyes: markerless mobile augmented reality for radiation awareness in the hybrid room. IEEE Trans Biomed Eng. 2017;64:429–40.CrossRefGoogle Scholar
  5. 5.
    Doug’s great demo 1968 [Internet]. Available from:
  6. 6.
    Over 50 years of Moore’s law [Internet]. Available from:
  7. 7.
    Milgram P, Kishino F. A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst. 1994;E77-D(12):1321–9.Google Scholar
  8. 8.
    What is mixed reality? [Internet]. Available from:
  9. 9.
    HoloAnatomy app wins top honors [Internet]. Available from:
  10. 10.
    Peterson DC, Mlynarczyk GSA. Analysis of traditional versus three-dimensional augmented curriculum on anatomical learning outcome measures. Anat Sci Educ. 2016;9:529–36.
  11. 11.
    Chan F, Aguirre S, Bauser H. Head tracked stereoscopic pre-surgical evaluation of major aortopulmonary collateral arteries in the newborns. Chicago: Paper presented at: Radiological Society of North America 2013 Scientific Assembly and Annual Meeting; 2013.Google Scholar
  12. 12.
    Jang J, Tschabrunn CM, Barkagan M, Anter E. Three-dimensional holographic visualization of high-resolution myocardial scar on HoloLens. PLoS One. 2018;13:e0205188.
  13. 13.
    Kiss G, Palmer CL, Torp H. Patient adapted augmented reality system for real-time echocardiographic applications. In: Linte CA, Yaniv Z, Fallavollita P, editors. Augmented environments for computer-assisted interventions. Cham: Springer International Publishing; 2015. p. 145–54.CrossRefGoogle Scholar
  14. 14.
    Opolski MP, Debski A, Borucki BA, Szpak M, Staruch AD, Kepka C, et al. First-in-man computed tomography-guided percutaneous revascularization of coronary chronic total occlusion using a wearable computer: proof of concept. Can J Cardiol. 2016;32:11–3.CrossRefGoogle Scholar
  15. 15.
    Katz A, Shtub A, Roguin A. Minimizing ionizing radiation exposure in invasive cardiology safety training for medical doctors. J Nucl Eng Radiat Sci. 2017;3:030905.CrossRefGoogle Scholar
  16. 16.
    Linte CA, Moore J, Wedlake C, Bainbridge D, Guiraudon GM, Jones DL, et al. Inside the beating heart: an in vivo feasibility study on fusing pre- and intra-operative imaging for minimally invasive therapy. Int J Comput Assist Radiol Surg. 2009;4:113–23.CrossRefGoogle Scholar
  17. 17.
    Moro C, Štromberga Z, Raikos A, Stirling A. The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anat Sci Educ. 2017;10:549–59.CrossRefGoogle Scholar
  18. 18.
    Statt N. Realmax’s prototype AR goggles fix one of the HoloLens’ biggest issues. Available from:
  19. 19.
    Vukicevic M, Mosadegh B, Min JK, Little SH. Cardiac 3D printing and its future directions. JACC Cardiovasc Imaging. 2017;10:171–84.
  20. 20.
    van der MOAJ, Schijven MP. The value of haptic feedback in conventional and robot-assisted minimal invasive surgery and virtual reality training: a current review. Surg Endosc. 2009;23:1180–90.CrossRefGoogle Scholar
  21. 21.
    King C-H, Culjat MO, Franco ML, Lewis CE, Dutson EP, Grundfest WS, et al. Tactile feedback induces reduced grasping force in robot-assisted surgery. IEEE Trans Haptics. 2009;2:103–10.CrossRefGoogle Scholar
  22. 22.
    Stetten G, Wu B, Klatzky R, Galeotti J, Siegel M, Lee R, et al. Hand-held force magnifier for surgical instruments 2011;90–100.Google Scholar
  23. 23.
    • Silva JNA, Southworth M, Raptis C, Silva J. Emerging applications of virtual reality in cardiovascular medicine. JACC Basic Transl Sci. 2018;3:420–30. Our previous review of this subject includes additional information about extended reality display hardware.CrossRefGoogle Scholar
  24. 24.
    • Peters TM, Linte CA, Yaniv Z, Williams J. Mixed and Augmented Reality in Medicine: CRC Press; 2018. Readers interested in a thorough treatment of XR in medical practice should consider this book.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Biomedical EngineeringWashington University in St Louis School of Engineering and Applied ScienceSt. LouisUSA
  2. 2.SentiAR, IncSt. LouisUSA
  3. 3.Department of Pediatrics, Division of CardiologyWashington University in St Louis School of MedicineSt LouisUSA

Personalised recommendations