Real-Time 3D Ultrasound Reconstruction and Visualization in the Context of Laparoscopy

  • Uditha L. JayarathneEmail author
  • John Moore
  • Elvis C. S. Chen
  • Stephen E. Pautler
  • Terry M. Peters
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10434)


In the context of laparoscopic interventions involving intracorporeal ultrasound, we present a method to visualize hidden targets in 3D. As the surgeon scans the organ surface, we stitch tracked 2D ultrasound images into a 3D volume in real-time. This volume, registered in space with the surface view provided by the laparoscope, is visualized through a transparent window in the surface image. The efficacy of the proposed method is demonstrated by conducting a psychophysical study with phantoms, involving experienced ultrasound users and laparoscopic surgeons. The results reveal that the proposed method demands significantly less cognitive and physical effort compared to the 2D ultrasound visualization method conventionally used in the operating room.


Laparoscopic ultrasound In-situ visualization Augmented-reality 3D ultrasound reconstruction Direct volume rendering 


  1. 1.
    Ameri, G., McLeod, A.J., Baxter, J.S.H., Chen, E.C.S., Peters, T.M.: Line fiducial material and thickness considerations for ultrasound calibration. In: Proceedings of SPIE Medical Imaging (2015)Google Scholar
  2. 2.
    Bichlmeier, C., Wimmer, F., Heining, S.M., Navab, N.: Contextual anatomic mimesis hybrid in-situ visualization method for improving multi-sensory depth perception in medical augmented reality. In: Proceedings of IEEE Symposium on Mixed and Augmented Reality, pp. 1–10 (2007)Google Scholar
  3. 3.
    Cheung, C.L., Wedlake, C., Moore, J., Pautler, S.E., Peters, T.M.: Fused video and ultrasound images for minimally invasive partial nephrectomy: a phantom study. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6363, pp. 408–415. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15711-0_51CrossRefGoogle Scholar
  4. 4.
    Garrett, W., Fuchs, H., Whitton, M., State, A.: Real-time incremental visualization of dynamic ultrasound volumes using parallel BSP trees. In: Proceedings of IEEE Visualization 1996, pp. 235–240 (1996)Google Scholar
  5. 5.
    Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv. Psychol. 52, 139–183 (1988)CrossRefGoogle Scholar
  6. 6.
    Hughes-Hallett, A., Pratt, P., Mayer, E., Di Marco, A., Yang, G.Z., Vale, J., Darzi, A.: Intraoperative ultrasound overlay in robot-assisted partial nephrectomy: first clinical experience. Eur. Urol. 65(3), 671–672 (2014)CrossRefGoogle Scholar
  7. 7.
    Jayarathne, U.L., McLeod, A.J., Peters, T.M., Chen, E.C.S.: Robust intraoperative US probe tracking using a monocular endoscopic camera. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8151, pp. 363–370. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40760-4_46CrossRefGoogle Scholar
  8. 8.
    Klatzky, R.L., Wu, B., Stetten, G.: Spatial representations from perception and cognitive mediation: the case of ultrasound. Curr. Dir. Psychol. Sci. 17(6), 359–364 (2008)CrossRefGoogle Scholar
  9. 9.
    Lasso, A., Heffter, T., Rankin, A., Pinter, C., Ungi, T., Fichtinger, G.: PLUS: open-source toolkit for ultrasound-guided intervention systems. IEEE Trans. Biomed. Eng. 61(10), 2527–2537 (2014)CrossRefGoogle Scholar
  10. 10.
    Lerotic, M., Chung, A.J., Mylonas, G., Yang, G.-Z.: pq-space based non-photorealistic rendering for augmented reality. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007. LNCS, vol. 4792, pp. 102–109. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-75759-7_13CrossRefGoogle Scholar
  11. 11.
    Ludvigsen, H.: Real-time GPU-based 3D ultrasound reconstruction and visualization. Master’s thesis, Norwegion University of Science and Technology (2010)Google Scholar
  12. 12.
    Pratt, P., Marco, A., Payne, C., Darzi, A., Yang, G.-Z.: Intraoperative ultrasound guidance for transanal endoscopic microsurgery. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7510, pp. 463–470. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33415-3_57CrossRefGoogle Scholar
  13. 13.
    Shepard, R.N., Metzler, J.: Mental rotation of three-dimensional objects. Science 171(3972), 701–703 (1971)CrossRefGoogle Scholar
  14. 14.
    Trobaugh, J.W., Trobaugh, D.J., Richard, W.D.: Three-dimensional imaging with stereotactic ultrasonography. Comput. Med. Imaging Graph. 18(5), 315–323 (1994)CrossRefGoogle Scholar
  15. 15.
    Wu, B., Klatzky, R.L., Stetten, G.: Visualizing 3D objects from 2D cross sectional images displayed in-situ versus ex-situ. J. Exp. Psychol. 16(1), 45–59 (2010)Google Scholar
  16. 16.
    Zhang, Z., Member, S.: A flexible new technique for camera calibration. IEEE Trans. PAMI 22(11), 1330–1334 (2000)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Uditha L. Jayarathne
    • 1
    • 2
    Email author
  • John Moore
    • 1
  • Elvis C. S. Chen
    • 1
    • 2
  • Stephen E. Pautler
    • 3
  • Terry M. Peters
    • 1
    • 2
  1. 1.Robarts Research InstituteLondonCanada
  2. 2.Department of Biomedical EngineeringWestern UniversityLondonCanada
  3. 3.Department of SurgeryWestern UniversityLondonCanada

Personalised recommendations