Direct Visual and Haptic Volume Rendering of Medical Data Sets for an Immersive Exploration in Virtual Reality

  • Balázs FaludiEmail author
  • Esther I. Zoller
  • Nicolas Gerig
  • Azhar Zam
  • Georg Rauter
  • Philippe C. Cattin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11768)


Visual examination of volumetric medical data sets in virtual reality offers an intuitive and immersive experience. To further increase the realism of virtual environments, haptic feedback can be added. Such systems can help students to gain anatomical knowledge or surgeons to prepare for specific interventions. In this work, we present a method for direct visual and haptic rendering of volumetric medical data sets in virtual reality. This method guarantees a continuous force field and does not rely on any mesh or surface generation. Using a transfer function, we mapped computed tomography voxel intensities to color and opacity values and then visualized the anatomical structures using a direct volume rendering approach. A continuous haptic force field was generated based on a conservative potential field computed from the voxel opacities. In a path following experiment, we showed that the deviation from a reference path on the surface of the rendered anatomical structure decreased with the added haptic feedback. This system demonstrates an immersive exploration of anatomy and is a step towards patient-specific surgical planning and simulation.


Haptic rendering CT Human-robot interaction Medical simulation Surgical planning 



This work was financially supported by the Werner Siemens Foundation through the MIRACLE project.

Supplementary material

490279_1_En_4_MOESM1_ESM.pdf (6.4 mb)
Supplementary material 1 (pdf 6548 KB)


  1. 1.
    Alaraj, A., et al.: Virtual reality cerebral aneurysm clipping simulation with real-time haptic feedback. Op. Neurosurg. 11(1), 52–58 (2015)Google Scholar
  2. 2.
    Andermatt, S., Pezold, S., Cattin, P.C.: Automated segmentation of multiple sclerosis lesions using multi-dimensional gated recurrent units. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 31–42. Springer, Cham (2018). Scholar
  3. 3.
    Avila, R.S., Sobierajski, L.M.: A haptic interaction method for volume visualization. In: Proceedings of Seventh Annual IEEE Visualization 1996, pp. 197–204. IEEE (1996)Google Scholar
  4. 4.
    Conti, F., et al.: The CHAI libraries. In: Proceedings of Eurohaptics 2003, Dublin, Ireland, pp. 496–500 (2003)Google Scholar
  5. 5.
    García-Lorenzo, D., Francis, S., Narayanan, S., Arnold, D.L., Collins, D.L.: Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging. Med. Image Anal. 17(1), 1–18 (2013)CrossRefGoogle Scholar
  6. 6.
    Laycock, S.D., Day, A.: A survey of haptic rendering techniques. In: Computer Graphics Forum, vol. 26, pp. 50–65. Wiley Online Library (2007)Google Scholar
  7. 7.
    Levoy, M.: Display of surfaces from volume data. IEEE Comput. Graph. Appl. 8(3), 29–37 (1988)CrossRefGoogle Scholar
  8. 8.
    Levoy, M.: Efficient ray tracing of volume data. ACM Trans. Graph. (TOG) 9(3), 245–261 (1990)CrossRefGoogle Scholar
  9. 9.
    Maloca, P.M., et al.: High-performance virtual reality volume rendering of original optical coherence tomography point-cloud data enhanced with real-time ray casting. Transl. Vis. Sci. Technol. 7(4), 2–2 (2018)CrossRefGoogle Scholar
  10. 10.
    Pfandler, M., Lazarovici, M., Stefan, P., Wucherer, P., Weigl, M.: Virtual reality-based simulators for spine surgery: a systematic review. Spine J. 17(9), 1352–1363 (2017)CrossRefGoogle Scholar
  11. 11.
    Rauter, G., Sigrist, R., Riener, R., Wolf, P.: Learning of temporal and spatial movement aspects: a comparison of four types of haptic control and concurrent visual feedback. IEEE Trans. Haptics 8(4), 421–433 (2015)CrossRefGoogle Scholar
  12. 12.
    Ren, J., Patel, R.V., McIsaac, K.A., Guiraudon, G., Peters, T.M.: Dynamic 3-d virtual fixtures for minimally invasive beating heart procedures. IEEE Trans. Med. Imaging 27(8), 1061–1070 (2008)CrossRefGoogle Scholar
  13. 13.
    Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust. Speech Signal Process. 26(1), 43–49 (1978)CrossRefGoogle Scholar
  14. 14.
    Wendland, H.: Piecewise polynomial, positive definite and compactly supported radial functions of minimal degree. Adv. Comput. Math. 4(1), 389–396 (1995)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Yagel, R., Shi, Z.: Accelerating volume animation by space-leaping. In: Proceedings of the 4th conference on Visualization 1993, pp. 62–69. IEEE Computer Society (1993)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Balázs Faludi
    • 1
    Email author
  • Esther I. Zoller
    • 2
  • Nicolas Gerig
    • 2
  • Azhar Zam
    • 3
  • Georg Rauter
    • 2
  • Philippe C. Cattin
    • 1
  1. 1.CIAN, Department of Biomedical EngineeringUniversity of BaselBaselSwitzerland
  2. 2.BIROMED-Lab, Department of Biomedical EngineeringUniversity of BaselBaselSwitzerland
  3. 3.BLOG, Department of Biomedical EngineeringUniversity of BaselBaselSwitzerland

Personalised recommendations