Abstract
Augmented Reality (AR) has already proven its worth in various applications in the medical domain. However, most of the solutions proposed were bound to specific hardware or software configurations, and/or their application was limited to specific cases, thus lacking in flexibility. In this paper, we present a software framework suitable for AR video see-through systems conceived for medical applications: our solution allows merging of real world images grabbed by one or more external cameras with computer-generated sceneries coregistered to the acquired images. The software framework is highly configurable and extensible thanks to the employment of two text configuration files that make it suitable for many typologies of potential applications. The proposed solution can be easily adapted to functioning with different tracking and AR visualization modalities. The versatility of the software for video see-through AR applications was already tested on various medical applications, in conjunction with head-mounted displays or with external spatial displays.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Peters, T.M.: Image-guided surgery: from X-rays to virtual reality. Comput. Meth. Biomechan. Biomed. Eng. 4, 27–57 (2000)
Peters, T.M.: Image-guidance for surgical procedures. Phys. Med. Biol. 51, R505–R540 (2006)
Kersten-Oertel, M., Jannin, P., Collins, D.L.: DVV: a taxonomy for mixed reality visualization in image guided surgery. IEEE Trans. Vis. Comput. Graph. 18, 332–352 (2012)
Navab, N., Traub, J., Sielhorst, T., Feuerstein, M., Bichlmeier, C.: Action- and workflow-driven augmented reality for computer-aided medical procedures. IEEE Comput. Graph. 27, 10–14 (2007)
Sielhorst, T., Feuerstein, M., Navab, N.: Advanced medical displays: a literature review of augmented reality. J. Disp. Technol. 4, 451–467 (2008)
Holloway, R.L.: Registration error analysis for augmented reality. Presence: Teleoperators Virtual Environ. 6, 413–432 (1997)
Rolland, J.P., Holloway, R.L., Fuchs, H.: A comparison of optical and video see-through head-mounted displays. Telemanipulator Telepresence Technol. 2351, 293–307 (1994)
Kellner, F., Bolte, B., Bruder, G., Rautenberg, U., Steinicke, F., Lappe, M., Koch, R.: Geometric calibration of head-mounted displays and its effects on distance estimation. IEEE Trans. Visual Comput. Graph. 18, 589–596 (2012)
Genc, Y., Sauer, F., Wenzel, F., Tuceryan, M., Navab, N.: Optical see-through HMD calibration: a stereo method validated with a video see-through system. In: Proceedings of the IEEE and ACM International Symposium on Augmented Reality, pp. 165–174 (2000)
Plopski, A., Itoh, Y., Nitschke, C., Kiyokawa, K., Klinker, G., Takemura, H.: Corneal-imaging calibration for optical see-through head-mounted displays. IEEE Trans. Vis. Comput. Graph. 21, 481–490 (2015)
Zhang, Z.Y.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. 22, 1330–1334 (2000)
Navab, N., Heining, S.M., Traub, J.: Camera Augmented Mobile C-Arm (CAMC): calibration, accuracy study, and clinical applications. IEEE Trans. Med. Imaging 29, 1412–1423 (2010)
Cutolo, F., Parchi, P.D., Ferrari, V.: Video see through AR head-mounted display for medical procedures. In: International Symposium on Mixed and Augmented Reality, pp. 393–396 (2014)
Hu, L., Wang, M.N., Song, Z.J.: A convenient method of video see-through augmented reality based on image-guided surgery system. In: 2013 Seventh International Conference on Internet Computing for Engineering and Science (ICICSE 2013), pp. 100–103 (2013)
Sauer, F., Vogt, S., Khamene, A., Heining, S., Euler, E., Schneberger, M., Zuerl, K., Mutschler, W.: Augmented reality visualization for thoracoscopic spine surgery. In: Proceedings of the Society of Photo-Optical Instrumentation, vol. 6141, p. 14106 (2006). Article no. 614106
Ferrari, V., Cutolo, F., Calabro, E.M., Ferrari, M.: HMD video see though AR with unfixed cameras vergence. In: International Symposium on Mixed and Augmented Reality, pp. 265–266 (2014)
Maria Viglialoro, R., Condino, S., Gesi, M., Ferrari, M., Ferrari, V.: Augmented reality simulator for laparoscopic cholecystectomy training. In: De Paolis, L.T., Mongelli, A. (eds.) AVR 2014. LNCS, vol. 8853, pp. 428–433. Springer, Heidelberg (2014)
Viglialoro, R., Condino, S., Freschi, C., Cutolo, F., Gesi, M., Ferrari, M., Ferrari, V.: Proceedings of the Twelfth IASTED International Conference on Biomedical Engineering. ACTA Press (2006)
Cutolo, F., Badiali, G., Ferrari, V.: Human-PnP: ergonomic AR interaction paradigm for manual placement of rigid bodies. In: Linte, C., Yaniv, Z., Fallavollita, P. (eds.) AE-CAI 2015. LNCS, vol. 9365, pp. 50–60. Springer, Heidelberg (2015)
Badiali, G., Ferrari, V., Cutolo, F., Freschi, C., Caramella, D., Bianchi, A., Marchetti, C.: Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J. Cranio Maxillofac. Surg. 42, 1970–1976 (2014)
Parchi, P.D., Piolanti, N., Andreani, L., Cutolo, F., Freschi, C., Mascioli, S., Ferrari, M., Lisanti, M.: Potentialities of wearable augmented reality in orthopaedics. In: 14th CAOS International (2014)
Acknowledgments
This work was funded by the Italian Ministry of Health grant SThARS (Surgical training in identification and isolation of deformable tubular structures with hybrid Augmented Reality Simulation, 6/11/2014–5/11/2017). Grant “Ricerca finalizzata e Giovani Ricercatori 2011-2012” Young Researchers – Italian Ministry of Health.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Cutolo, F., Siesto, M., Mascioli, S., Freschi, C., Ferrari, M., Ferrari, V. (2016). Configurable Software Framework for 2D/3D Video See-Through Displays in Medical Applications. In: De Paolis, L., Mongelli, A. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2016. Lecture Notes in Computer Science(), vol 9769. Springer, Cham. https://doi.org/10.1007/978-3-319-40651-0_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-40651-0_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-40650-3
Online ISBN: 978-3-319-40651-0
eBook Packages: Computer ScienceComputer Science (R0)