Abstract
In the context of Augmented Reality (AR), industrial pioneers and early adopters have considered the wearable optical see-through (OST) displays as proper and effective tools in applications spanning from manufacturing and maintenance up to the entertainment field and the medical area, because they provide the user with an egocentric viewpoint maintaining the quality of the visual perception of the real world.
The common OST displays paradigm entails intrinsic perceptual conflicts owing to mismatched accommodation between real 3D world and virtual 2D images projected over semitransparent surfaces. Such paradigm is suitable for augmenting the reality with simple virtual elements (models, icons or text), but various shortcomings remain in case of complex virtual contents. The major shortcoming is due to the tedious and error prone calibration methods required to obtain geometrical consistency, pivotal in many of the aforementioned fields of application. These shortcomings are due to the intrinsic incompatibility between the nature of the 4D light field, related to the real world, and the nature of the virtual content, rendered as a 2D image.
In this paper we describe a radical rethinking of the wearable OST displays paradigm by generating, through integral imaging technique, the virtual content as a light field, in order to overcome the typical limitations of the traditional approach. This paper describes the hardware components and an innovative rendering strategy in more details in respect to a previous work. Furthermore we report early results with the implementation of the integral imaging display using a lens array instead of a pinhole array.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Badiali, G., Ferrari, V., Cutolo, F., Freschi, C., Caramella, D., Bianchi, A., Marchetti, C.: Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J. Craniomaxillofac. Surg. 42, 1970–1976 (2014)
Kersten-Oertel, M., Jannin, P., Collins, D.L.: The state of the art of visualization in mixed reality image guided surgery. Comput. Med. Imaging Graph. 37, 98–112 (2013)
Cutolo, F., Badiali, G., Ferrari, V.: Human-PnP: ergonomic AR interaction paradigm for manual placement of rigid bodies. In: Linte, C.A., Yaniv, Z., Fallavollita, P. (eds.) AE-CAI 2015. LNCS, vol. 9365, pp. 50–60. Springer, Cham (2015). doi:10.1007/978-3-319-24601-7_6
Rankin, T.M., Slepian, M.J., Armstrong, D.G.: Augmented reality in surgery. In: Latifi, R., Rhee, P., Gruessner, W.G.R. (eds.) Technological Advances in Surgery, Trauma and Critical Care, pp. 59–71. Springer, New York (2015)
Ferrari, V., Cutolo, F., Calabrò, E.M., Ferrari, M.: HMD video see through AR with unfixed cameras vergence. In: International Symposium on Mixed and Augmented Reality – ISMAR (2014)
Cutolo, F., Parchi, P.D., Ferrari, V.: Video see through AR head-mounted display for medical procedures. In: International Symposium on Mixed and Augmented Reality – ISMAR (2014)
Ferrari, V., Megali, G., Troia, E., Pietrabissa, A., Mosca, F.: A 3-D mixed-reality system for stereoscopic visualization of medical dataset. IEEE Trans. Biomed. Eng. 56, 2627–2633 (2009)
Meola, A., Cutolo, F., Carbone, M., Cagnazzo, F., Ferrari, M., Ferrari, V.: Augmented reality in neurosurgery: a systematic review. Neurosurg. Rev. 1–12 (2016). doi:10.1007/s10143-016-0732-9 [epub ahead of print]
Abe, Y., Sato, S., Kato, K., Hyakumachi, T., Yanagibashi, Y., Ito, M., Abumi, K.: A novel 3D guidance system using augmented reality for percutaneous vertebroplasty. J. Neurosurg. Spine 19, 492–501 (2013)
Ferrari, V., Ferrari, M., Mosca, F.: Video see-through in the clinical practice. In: 1st International Workshop on Engineering Interactive Computing Systems for Medicine and Health Care, EICS4Med, vol. 727, pp. 19–24 (2011)
Ferrari, V.: Letter to the editor on “designing a wearable navigation system for image-guided cancer resection surgery”. Ann. Biomed. Eng. 42, 2600–2601 (2014)
Cutolo, F., Carbone, M., Parchi, P.D., Ferrari, V., Lisanti, M., Ferrari, M.: Application of a new wearable augmented reality video see-through display to aid percutaneous procedures in spine surgery. In: Paolis, L.T., Mongelli, A. (eds.) AVR 2016. LNCS, vol. 9769, pp. 43–54. Springer, Cham (2016). doi:10.1007/978-3-319-40651-0_4
Parrini, S., Cutolo, F., Freschi, C., Ferrari, M., Ferrari, V.: Augmented reality system for freehand guide of magnetic endovascular devices. In: Conference Proceedings of IEEE Engineering in Medicine and Biology Society 2014, pp. 490–493 (2014)
Sielhorst, T., Feuerstein, M., Navab, N.: Advanced medical displays: a literature review of augmented reality. J. Disp. Technol. 4, 451–467 (2008)
Holloway, R.L.: Registration error analysis for augmented reality. Presence Teleop. Virt. 6, 413–432 (1997)
Benton, S.A.: Selected Papers on Three-Dimensional Displays. SPIE Optical Engineering Press, Bellingham (2001)
Rolland, J.P., Cakmakci, O.: The past, present, and future of head-mounted display designs, pp. 368–377 (2005)
Holliman, N.S., Dodgson, N.A., Favalora, G.E., Pockett, L.: Three-dimensional displays: a review and applications analysis. IEEE Trans. Broadcast. 57, 362–371 (2011)
Mukawa, H., Akutsu, K., Matsumura, I., Nakano, S., Yoshida, T., Kuwahara, M., Aiki, K., Ogawa, M.: Distinguished paper: a full color eyewear display using holographic planar waveguides. SID Int. Symp. Dig. Tech. Papers 39, 89–92 (2008)
Rolland, J.P., Holloway, R.L., Fuchs, H.: A comparison of optical and video see-through head-mounted displays. Telemanipulator Telepresence Technol. 2351, 293–307 (1994)
Tuceryan, M., Genc, Y., Navab, N.: Single-point active alignment method (SPAAM) for optical see-through HMD calibration for augmented reality. Presence Teleop. Virt. 11, 259–276 (2002)
Genc, Y., Tuceryan, M., Navab, N.: Practical solutions for calibration of optical see-through devices. In: International Symposium on Mixed and Augmented Reality, Proceedings, pp. 169–175 (2002)
Gilson, S.J., Fitzgibbon, A.W., Glennerster, A.: Spatial calibration of an optical see-through head-mounted display. J. Neurosci. Meth. 173, 140–146 (2008)
Kellner, F., Bolte, B., Bruder, G., Rautenberg, U., Steinicke, F., Lappe, M., Koch, R.: Geometric calibration of head-mounted displays and its effects on distance estimation. IEEE Trans. Vis. Comput. Graph. 18, 589–596 (2012)
Plopski, A., Itoh, Y., Nitschke, C., Kiyokawa, K., Klinker, G., Takemura, H.: Corneal-imaging calibration for optical see-through head-mounted displays. IEEE Trans. Vis. Comput. Graph. 21, 481–490 (2015)
Levoy, M.: Light fields and computational imaging. Computer 39, 46–55 (2006)
Hua, H., Javidi, B.: A 3D integral imaging optical see-through head-mounted display. Opt. Express 22, 13484–13491 (2014)
Ferrari, V., Calabrò, E.M.: Wearable light field optical see-through display to avoid user dependent calibrations: a feasibility study. In: SAI Computing Conference (2016)
Liao, H., Inomata, T., Sakuma, I., Dohi, T.: 3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans. Biomed. Eng. 57, 1476–1486 (2010)
Stern, A., Javidi, B.: Three-dimensional image sensing, visualization, and processing using integral imaging. Proc. IEEE 94, 591–607 (2006)
Lippmann, G.: Epreuves reversibles donnant la sensation du relief. J. Phys. Theor. Appl. 7, 821–825 (1908)
Park, S.-G., Yeom, J., Jeong, Y., Chen, N., Hong, J.-Y., Lee, B.: Recent issues on integral imaging and its applications. J. Inf. Disp. 15, 37–46 (2014)
Wang, J., Suenaga, H., Liao, H., Hoshi, K., Yang, L., Kobayashi, E., Sakuma, I.: Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation. Comput. Med. Imaging Graph. 40, 147–159 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Calabrò, E.M., Cutolo, F., Carbone, M., Ferrari, V. (2017). Wearable Augmented Reality Optical See Through Displays Based on Integral Imaging. In: Perego, P., Andreoni, G., Rizzo, G. (eds) Wireless Mobile Communication and Healthcare. MobiHealth 2016. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 192. Springer, Cham. https://doi.org/10.1007/978-3-319-58877-3_44
Download citation
DOI: https://doi.org/10.1007/978-3-319-58877-3_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-58876-6
Online ISBN: 978-3-319-58877-3
eBook Packages: Computer ScienceComputer Science (R0)