Mobile, real-time, and point-of-care augmented reality is robust, accurate, and feasible: a prospective pilot study
Augmented reality (AR) systems are currently being explored by a broad spectrum of industries, mainly for improving point-of-care access to data and images. Especially in surgery and especially for timely decisions in emergency cases, a fast and comprehensive access to images at the patient bedside is mandatory. Currently, imaging data are accessed at a distance from the patient both in time and space, i.e., at a specific workstation. Mobile technology and 3-dimensional (3D) visualization of radiological imaging data promise to overcome these restrictions by making bedside AR feasible.
In this project, AR was realized in a surgical setting by fusing a 3D-representation of structures of interest with live camera images on a tablet computer using marker-based registration. The intent of this study was to focus on a thorough evaluation of AR. Feasibility, robustness, and accuracy were thus evaluated consecutively in a phantom model and a porcine model. Additionally feasibility was evaluated in one male volunteer.
In the phantom model (n = 10), AR visualization was feasible in 84% of the visualization space with high accuracy (mean reprojection error ± standard deviation (SD): 2.8 ± 2.7 mm; 95th percentile = 6.7 mm). In a porcine model (n = 5), AR visualization was feasible in 79% with high accuracy (mean reprojection error ± SD: 3.5 ± 3.0 mm; 95th percentile = 9.5 mm). Furthermore, AR was successfully used and proved feasible within a male volunteer.
Mobile, real-time, and point-of-care AR for clinical purposes proved feasible, robust, and accurate in the phantom, animal, and single-trial human model shown in this study. Consequently, AR following similar implementation proved robust and accurate enough to be evaluated in clinical trials assessing accuracy, robustness in clinical reality, as well as integration into the clinical workflow. If these further studies prove successful, AR might revolutionize data access at patient bedside.
KeywordsAugmented reality Mobile device Image visualization Visual assistance
The current study was conducted within the setting of the Research Training Group 1126 (“Development of New Computer-Based Methods for the Future Workplace in Surgery”) and the Collaborative Research Center 125 (“Cognition Guided Surgery”); both were funded by the German Research Foundation.
Compliance with ethical standards
Drs. Hannes Götz Kenngott, Anas Amin Preukschas, Martin Wagner, Felix Nickel, Michael Müller, Nadine Bellemann, Christian Stock, Markus Fangerau, Boris Radeleff, Hans-Ulrich Kauczor, Hans-Peter Meinzer, Lena Maier-Hein, and Beat Peter Müller-Stich have no conflicts of interest or financial ties to disclose.
Supplementary material 1 (MP4 29711 KB)
- 1.Bellini HC, Sugiyama M, Shin M, Alam S, Takayama D (2016) Virtual & augmented reality understanding the race for the next computing platform, Jan 13, 2016 edn. The Goldman Sachs Group, Inc., New York CityGoogle Scholar
- 5.Kay M, Santos J, Takane M (2011) mHealth: new horizons for health through mobile technologies. World Health Organization, GenevaGoogle Scholar
- 6.Mecheal PSD (2008) Towards the development of an mHealth strategy: a literature review. In: World Health Organization TMVPaTEIaCU. World Health Organization, GenevaGoogle Scholar
- 14.Kenngott HG, Wunscher JJ, Wagner M, Preukschas A, Wekerle AL, Neher P, Suwelack S, Speidel S, Nickel F, Oladokun D, Maier-Hein L, Dillmann R, Meinzer HP, Muller-Stich BP (2015) OpenHELP (Heidelberg laparoscopy phantom): development of an open-source surgical evaluation and training tool. Surg Endosc 29:3338–3347CrossRefPubMedPubMedCentralGoogle Scholar
- 15.Nolden M, Zelzer S, Seitel A, Wald D, Muller M, Franz AM, Maleike D, Fangerau M, Baumhauer M, Maier-Hein L, Maier-Hein KH, Meinzer HP, Wolf I (2013) The medical imaging interaction toolkit: challenges and advances: 10 years of open-source development. Int J Comput Assist Radiol Surg 8:607–620CrossRefPubMedGoogle Scholar
- 23.Kenngott HG, Wagner M, Gondan M, Nickel F, Nolden M, Fetzer A, Weitz J, Fischer L, Speidel S, Meinzer H-P, Böckler D, Büchler MW, Müller-Stich BP (2013) Real-time image guidance in laparoscopic liver surgery: first clinical experience with a guidance system based on intraoperative CT imaging. Surg Endosc 28:933–940CrossRefPubMedGoogle Scholar