Virtual Reality Surgery Simulation: A Survey on Patient Specific Solution

  • Jinglu Zhang
  • Jian Chang
  • Xiaosong Yang
  • Jian J. ZhangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10582)


For surgeons, the precise anatomy structure and its dynamics are important in the surgery interaction, which is critical for generating the immersive experience in VR based surgical training applications. Presently, a normal therapeutic scheme might not be able to be straightforwardly applied to a specific patient, because the diagnostic results are based on averages, which result in a rough solution. Patient Specific Modeling (PSM), using patient-specific medical image data (e.g. CT, MRI, or Ultrasound), could deliver a computational anatomical model. It provides the potential for surgeons to practice the operation procedures for a particular patient, which will improve the accuracy of diagnosis and treatment, thus enhance the prophetic ability of VR simulation framework and raise the patient care. This paper presents a general review based on existing literature of patient specific surgical simulation on data acquisition, medical image segmentation, computational mesh generation, and soft tissue real time simulation.


Patient Specific Modeling Surgery simulation Virtual reality 



We would also like to thank the People Program (Marie Curie Actions) of the European Union’s Seventh Framework Program FP7/2007-2013/ under REA grant agreement n\(^{\circ }\) [612627] for their support.


  1. 1.
    Adams, R., Bischof, L.: Seeded region growing. IEEE Trans. Pattern Anal. Mach. Intell. 16(6), 641–647 (1994)CrossRefGoogle Scholar
  2. 2.
    Alliez, P., Cohen-Steiner, D., Yvinec, M., Desbrun, M.: Variational tetrahedral meshing. ACM Trans. Graph. (TOG) 24, 617–625 (2005). ACMCrossRefGoogle Scholar
  3. 3.
    Antiga, L., Piccinelli, M., Botti, L., Ene-Iordache, B., Remuzzi, A., Steinman, D.A.: An image-based modeling framework for patient-specific computational hemodynamics. Med. Biol. Eng. Comput. 46(11), 1097 (2008)CrossRefGoogle Scholar
  4. 4.
    Badash, I., Burtt, K., Solorzano, C.A., Carey, J.N.: Innovations in surgery simulation: a review of past, current and future techniques. Ann. Transl. Med. 4(23), 453 (2016)CrossRefGoogle Scholar
  5. 5.
    Baraff, D., Witkin, A.: Large steps in cloth simulation. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 43–54. ACM (1998)Google Scholar
  6. 6.
    Barratt, D.C., Chan, C.S., Edwards, P.J., Penney, G.P., Slomczykowski, M., Carter, T.J., Hawkes, D.J.: Instantiation and registration of statistical shape models of the femur and pelvis using 3d ultrasound imaging. Med. Image Anal. 12(3), 358–374 (2008)CrossRefGoogle Scholar
  7. 7.
    Bender, J., Koschier, D., Charrier, P., Weber, D.: Position-based simulation of continuous materials. Comput. Graph. 44, 1–10 (2014)CrossRefGoogle Scholar
  8. 8.
    Boltcheva, D., Yvinec, M., Boissonnat, J.D.: Mesh generation from 3d multi-material images. In: Medical Image Computing and Computer-Assisted Intervention MICCAI 2009, pp. 283–290 (2009)Google Scholar
  9. 9.
    Bonet, J., Burton, A.: A simple average nodal pressure tetrahedral element for incompressible and nearly incompressible dynamic explicit applications. Int. J. Numer. Meth. Biomed. Eng. 14(5), 437–449 (1998)zbMATHMathSciNetGoogle Scholar
  10. 10.
    Bouaziz, S., Martin, S., Liu, T., Kavan, L., Pauly, M.: Projective dynamics: fusing constraint projections for fast simulation. ACM Trans. Graph. (TOG) 33(4), 154 (2014)CrossRefGoogle Scholar
  11. 11.
    Bryson, S.: Virtual reality in scientific visualization. Commun. ACM 39(5), 62–71 (1996)CrossRefGoogle Scholar
  12. 12.
    Cevidanes, L.H., Tucker, S., Styner, M., Kim, H., Chapuis, J., Reyes, M., Proffit, W., Turvey, T., Jaskolka, M.: Three-dimensional surgical simulation. Am. J. Orthod. Dentofac. Orthop. 138(3), 361–371 (2010)CrossRefGoogle Scholar
  13. 13.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998). doi: 10.1007/BFb0054760 CrossRefGoogle Scholar
  14. 14.
    Cootes, T.F., Taylor, C.J.: Active shape models-‘smart snakes’. In: Hogg, D., Boyle, R. (eds.) BMVC 1992, pp. 266–275. Springer, London (1992)Google Scholar
  15. 15.
    Davis, J.E.: The use of simulation in causal analysis of sentinel events in healthcare. Ph.D. thesis, University of Pennsylvania (2016)Google Scholar
  16. 16.
    Duffy, A., Hogle, N., McCarthy, H., Lew, J., Egan, A., Christos, P., Fowler, D.: Construct validity for the LapSim laparoscopic surgical simulator. Surg. Endosc. Interv. Tech. 19(3), 401–405 (2005)CrossRefGoogle Scholar
  17. 17.
    Endo, K., Sata, N., Ishiguro, Y., Miki, A., Sasanuma, H., Sakuma, Y., Shimizu, A., Hyodo, M., Lefor, A., Yasuda, Y.: A patient-specific surgical simulator using preoperative imaging data: an interactive simulator using a three-dimensional tactile mouse. J. Comput. Surg. 1(1), 10 (2014)CrossRefGoogle Scholar
  18. 18.
    Eschweiler, J., Stromps, J.P., Fischer, M., Schick, F., Rath, B., Pallua, N., Radermacher, K.: A biomechanical model of the wrist joint for patient-specific model guided surgical therapy: part 2. Proc. Inst. Mech. Eng. Part H J. Eng. Med. 230(4), 326–334 (2016)CrossRefGoogle Scholar
  19. 19.
    Eschweiler, J., Stromps, J.P., Fischer, M., Schick, F., Rath, B., Pallua, N., Radermacher, K.: Development of a biomechanical model of the wrist joint for patient-specific model guided surgical therapy planning: part 1. Proc. Inst. Mech. Eng. Part H J. Eng. Med. 230(4), 310–325 (2016)CrossRefGoogle Scholar
  20. 20.
    Fedorov, A., Beichel, R., Kalpathy-Cramer, J., Finet, J., Fillion-Robin, J.C., Pujol, S., Bauer, C., Jennings, D., Fennessy, F., Sonka, M., et al.: 3d slicer as an image computing platform for the quantitative imaging network. Magn. Reson. Imag. 30(9), 1323–1341 (2012)CrossRefGoogle Scholar
  21. 21.
    Gallagher, A.G., Ritter, E.M., Champion, H., Higgins, G., Fried, M.P., Moses, G., Smith, C.D., Satava, R.M.: Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann. Surg. 241(2), 364–372 (2005)CrossRefGoogle Scholar
  22. 22.
    Goel, V.R., Greenberg, R.K., Greenberg, D.P.: Mathematical analysis of DICOM CT datasets: can endograft sizing be automated for complex anatomy? J. Vasc. Surg. 47(6), 1306–1312 (2008)CrossRefGoogle Scholar
  23. 23.
    Gonzalez, R., Wintz, P.: Digital Image Processing. Prentice-Hall, Englewood Cliffs (1977)zbMATHGoogle Scholar
  24. 24.
    Hartigan, J.A., Wong, M.A.: Algorithm as 136: a k-means clustering algorithm. J. Royal Stat. Soc. Ser. C (Applied Statistics) 28(1), 100–108 (1979)zbMATHGoogle Scholar
  25. 25.
    Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017)CrossRefGoogle Scholar
  26. 26.
    Heimann, T., Meinzer, H.P.: Statistical shape models for 3d medical image segmentation: a review. Med. Image Anal. 13(4), 543–563 (2009)CrossRefGoogle Scholar
  27. 27.
    Indira, S., Ramesh, A.: Image segmentation using artificial neural network and genetic algorithm: a comparative analysis. In: 2011 International Conference on Process Automation, Control and Computing (PACC), pp. 1–6. IEEE (2011)Google Scholar
  28. 28.
    Iwamoto, N., Shum, H.P., Yang, L., Morishima, S.: Multi-layer lattice model for real-time dynamic character deformation. Comput. Graph. Forum 34, 99–109 (2015). Wiley Online LibraryCrossRefGoogle Scholar
  29. 29.
    Jain, R., Kasturi, R., Schunck, B.G.: Machine Vision, vol. 5. McGraw-Hill, New York (1995)Google Scholar
  30. 30.
    Jamin, C., Alliez, P., Yvinec, M., Boissonnat, J.D.: CGALmesh: a generic framework for delaunay mesh generation. ACM Trans. Math. Softw. (TOMS) 41(4), 23 (2015)CrossRefzbMATHMathSciNetGoogle Scholar
  31. 31.
    Johnson, C.: Biomedical visual computing: case studies and challenges. Comput. Sci. Eng. 14(1), 12–21 (2012)CrossRefGoogle Scholar
  32. 32.
    Kent, D.M., Hayward, R.A.: Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. JAMA 298(10), 1209–1212 (2007)CrossRefGoogle Scholar
  33. 33.
    Lai, J.Y., Essomba, T., Lee, P.Y., et al.: Algorithm for segmentation and reduction of fractured bones in computer-aided preoperative surgery. In: Proceedings of the 3rd International Conference on Biomedical and Bioinformatics Engineering, pp. 12–18. ACM (2016)Google Scholar
  34. 34.
    Leea, C.K., Mihaib, L.A., Halec, J.S., Kerfridena, P., Bordasc, S.P.: Strain smoothing for compressible and nearly-incompressible finite elasticity. Comput. Struct. 182, 540–555 (2016)CrossRefGoogle Scholar
  35. 35.
    Lei, T., Sewchand, W.: Statistical approach to X-ray CT imaging and its applications in image analysis. II. A new stochastic model-based image segmentation technique for X-ray CT image. IEEE Trans. Med. Imag. 11(1), 62–69 (1992)CrossRefGoogle Scholar
  36. 36.
    Liu, T., Bargteil, A.W., O’Brien, J.F., Kavan, L.: Fast simulation of mass-spring systems. ACM Trans. Graph. (TOG) 32(6), 214 (2013)Google Scholar
  37. 37.
    Lorensen, W.E., Cline, H.E.: Marching cubes: a high resolution 3d surface construction algorithm. ACM SIGGRAPH Comput. Graph. 21, 163–169 (1987). ACMCrossRefGoogle Scholar
  38. 38.
    Makiyama, K., Nagasaka, M., Inuiya, T., Takanami, K., Ogata, M., Kubota, Y.: Development of a patient-specific simulator for laparoscopic renal surgery. Int. J. Urol. 19(9), 829–835 (2012)CrossRefGoogle Scholar
  39. 39.
    Marr, D., Hildreth, E.: Theory of edge detection. Proc. Royal Soc. Lond. B Biol. Sci. 207(1167), 187–217 (1980)CrossRefGoogle Scholar
  40. 40.
    Mihalef, V., Ionasec, R.I., Sharma, P., Georgescu, B., Voigt, I., Suehling, M., Comaniciu, D.: Patient-specific modelling of whole heart anatomy, dynamics and haemodynamics from four-dimensional cardiac CT images. Interface Focus 1(3), 286–296 (2011)CrossRefGoogle Scholar
  41. 41.
    Miller, K.: Biomechanics of Brain for Computer Integrated Surgery. Warsaw University of Technology Publishing House, Warsaw (2002)Google Scholar
  42. 42.
    Mohamed, A., Davatzikos, C.: Finite element mesh generation and remeshing from segmented medical images. In: 2004 IEEE International Symposium on Biomedical Imaging: Nano to Macro, pp. 420–423. IEEE (2004)Google Scholar
  43. 43.
    Müller, M., Heidelberger, B., Hennix, M., Ratcliff, J.: Position based dynamics. J. Vis. Commun. Image Represent. 18(2), 109–118 (2007)CrossRefGoogle Scholar
  44. 44.
    Neal, M.L., Kerckhoffs, R.: Current progress in patient-specific modeling. Briefings Bioinform. 11(1), 111–126 (2010)CrossRefGoogle Scholar
  45. 45.
    Nolden, M., Zelzer, S., Seitel, A., Wald, D., Müller, M., Franz, A.M., Maleike, D., Fangerau, M., Baumhauer, M., Maier-Hein, L., et al.: The medical imaging interaction toolkit: challenges and advances. Int. J. Comput. Assist. Radiol. Surg. 8(4), 607–620 (2013)CrossRefGoogle Scholar
  46. 46.
    de Oliveira, J.E., Giessler, P., Deserno, T.M.: Patient-specific anatomical modelling. In: E-Health and Bioengineering Conference (EHB), pp. 1–4. IEEE (2015)Google Scholar
  47. 47.
    O’Reilly, M.A., Whyne, C.M.: Comparison of computed tomography based parametric and patient-specific finite element models of the healthy and metastatic spine using a mesh-morphing algorithm. Spine 33(17), 1876–1881 (2008)CrossRefGoogle Scholar
  48. 48.
    Otaduy, M.A., Bickel, B., Bradley, D., Wang, H.: Data-driven simulation methods in computer graphics: cloth, tissue and faces. In: ACM SIGGRAPH 2012 Courses, p. 12. ACM (2012)Google Scholar
  49. 49.
    Pal, N.R., Pal, S.K.: A review on image segmentation techniques. Pattern Recognit. 26(9), 1277–1294 (1993)CrossRefGoogle Scholar
  50. 50.
    Pan, J.J., Chang, J., Yang, X., Liang, H., Zhang, J.J., Qureshi, T., Howell, R., Hickish, T.: Virtual reality training and assessment in laparoscopic rectum surgery. Int. J. Med. Rob. Comput. Assist. Surg. 11(2), 194–209 (2015)CrossRefGoogle Scholar
  51. 51.
    Prewitt, J.M.: Object enhancement and extraction. Picture Process. Psychopictorics 10(1), 15–19 (1970)Google Scholar
  52. 52.
    Ricotta, J.J., Pagan, J., Xenos, M., Alemu, Y., Einav, S., Bluestein, D.: Cardiovascular disease management: the need for better diagnostics. Med. Biol. Eng. Comput. 46(11), 1059–1068 (2008)CrossRefGoogle Scholar
  53. 53.
    Rineau, L., Yvinec, M.: Meshing 3d domains bounded by piecewise smooth surfaces. In: Brewer, M.L., Marcum, D. (eds.) Proceedings of the 16th International Meshing Roundtable, pp. 443–460. Springer, Heidelberg (2008)Google Scholar
  54. 54.
    Schöberl, J.: Netgen an advancing front 2d/3d-mesh generator based on abstract rules. Comput. Vis. Sci. 1(1), 41–52 (1997)CrossRefzbMATHGoogle Scholar
  55. 55.
    Sifakis, E., Barbic, J.: FEM simulation of 3d deformable solids: a practitioner’s guide to theory, discretization and model reduction. In: ACM SIGGRAPH 2012 Courses, p. 20. ACM (2012)Google Scholar
  56. 56.
    Viceconti, M., Davinelli, M., Taddei, F., Cappello, A.: Automatic generation of accurate subject-specific bone finite element models to be used in clinical studies. J. Biomech. 37(10), 1597–1605 (2004)CrossRefGoogle Scholar
  57. 57.
    Weatherill, N.P., Hassan, O.: Efficient three-dimensional delaunay triangulation with automatic point creation and imposed boundary constraints. Int. J. Numer. Meth. Eng. 37(12), 2005–2039 (1994)CrossRefzbMATHGoogle Scholar
  58. 58.
    Zhang, A., Hünerbein, M., Dai, Y., Schlag, P.M., Beller, S.: Construct validity testing of a laparoscopic surgery simulator (lap mentor®). Surg. Endosc. 22(6), 1440–1444 (2008)CrossRefGoogle Scholar
  59. 59.
    Zhang, Y., Hughes, T.J., Bajaj, C.L.: An automatic 3d mesh generation method for domains with multiple materials. Comput. Meth. Appl. Mech. Eng. 199(5), 405–415 (2010)CrossRefzbMATHGoogle Scholar
  60. 60.
    Zienkiewicz, O.C., Taylor, R.L.: The Finite Element Method for Solid and Structural Mechanics. Butterworth-Heinemann, Oxford (2005)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Jinglu Zhang
    • 1
  • Jian Chang
    • 1
  • Xiaosong Yang
    • 1
  • Jian J. Zhang
    • 1
    Email author
  1. 1.National Centre for Computer AnimationBournemouth UniversityPooleUK

Personalised recommendations