Abstract
Shape instantiation which predicts the 3D shape of a dynamic target from one or more 2D images is important for real-time intra-operative navigation. Previously, a general shape instantiation framework was proposed with manual image segmentation to generate a 2D Statistical Shape Model (SSM) and with Kernel Partial Least Square Regression (KPLSR) to learn the relationship between the 2D and 3D SSM for 3D shape prediction. In this paper, the two-stage shape instantiation is improved to be one-stage. PointOutNet with 19 convolutional layers and three fully-connected layers is used as the network structure and Chamfer distance is used as the loss function to predict the 3D target point cloud from a single 2D image. With the proposed one-stage shape instantiation algorithm, a spontaneous image-to-point cloud training and inference can be achieved. A dataset from 27 Right Ventricle (RV) subjects, indicating 609 experiments, were used to validate the proposed one-stage shape instantiation algorithm. An average point cloud-to-point cloud (PC-to-PC) error of 1.72 mm has been achieved, which is comparable to the PLSR-based (1.42 mm) and KPLSR-based (1.31 mm) two-stage shape instantiation algorithm.
X.-Y. Zhou and Z.-Y. Wang—Contribute equally to this paper. This work was supported by EPSRC project grant EP/L020688/1.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Choy, C.B., Xu, D., Gwak, J.Y., Chen, K., Savarese, S.: 3D-R2N2: a unified approach for single and multi-view 3D object reconstruction. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 628–644. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_38
Cool, D., Downey, D., Izawa, J., Chin, J., Fenster, A.: 3D prostate model formation from non-parallel 2D ultrasound biopsy images. Med. Image Anal. 10(6), 875–887 (2006)
Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3D object reconstruction from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 605–613 (2017)
Kulon, D., Wang, H., Güler, R.A., Bronstein, M., Zafeiriou, S.: Single image 3D hand reconstruction with mesh convolutions. arXiv preprint arXiv:1905.01326 (2019)
Lee, S.-L., Chung, A., Lerotic, M., Hawkins, M.A., Tait, D., Yang, G.-Z.: Dynamic shape instantiation for intra-operative guidance. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6361, pp. 69–76. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15705-9_9
Mandikal, P., Murthy, N., Agarwal, M., Babu, R.V.: 3D-LMNet: latent embedding matching for accurate and diverse 3D point cloud reconstruction from a single image. arXiv preprint arXiv:1807.07796 (2018)
Manu: nonrigidicp (2016). https://uk.mathworks.com/matlabcentral/fileexchange/41396-nonrigidicp. Accessed 02 Apr 2019
Toth, D., Pfister, M., Maier, A., Kowarschik, M., Hornegger, J.: Adaption of 3D models to 2D X-Ray images during endovascular abdominal aneurysm repair. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 339–346. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24553-9_42
Zheng, J.Q., Zhou, X.Y., Riga, C., Yang, G.Z.: 3D path planning from a single 2D fluoroscopic image for robot assisted fenestrated endovascular aortic repair. arXiv preprint arXiv:1809.05955 (2018)
Zheng, J.Q., Zhou, X.Y., Yang, G.Z.: Real-time 3D shape instantiation for partially-deployed stent segment from a single 2D fluoroscopic image in robot-assisted fenestrated endovascular aortic repair. arXiv preprint arXiv:1902.11089 (2019)
Zhou, X., Yang, G., Riga, C., Lee, S.: Stent graft shape instantiation for fenestrated endovascular aortic repair. In: The Hamlyn Symposium on Medical Robotics, pp. 78–79 (2016)
Zhou, X.Y., Lin, J., Riga, C., Yang, G.Z., Lee, S.L.: Real-time 3-D shape instantiation from single fluoroscopy projection for fenestrated stent graft deployment. IEEE Robot. Autom. Lett. 3(2), 1314–1321 (2018)
Zhou, X.Y., Yang, G.Z., Lee, S.L.: A real-time and registration-free framework for dynamic shape instantiation. Med. Image Anal. 44, 86–97 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, XY., Wang, ZY., Li, P., Zheng, JQ., Yang, GZ. (2019). One-Stage Shape Instantiation from a Single 2D Image to 3D Point Cloud. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11767. Springer, Cham. https://doi.org/10.1007/978-3-030-32251-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-32251-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32250-2
Online ISBN: 978-3-030-32251-9
eBook Packages: Computer ScienceComputer Science (R0)
