Abstract
We aim to elucidate the mechanism of the foot by automated measurement of its multiple bone movement using 2D-3D registration of bi-plane x-ray video and a stationary 3D CT. Conventional analyses allowed tracking of only 3 large proximal tarsal bones due to the requirement of manual segmentation and manual initialization of 2D-3D registration. The learning-based 2D-3D registration, on the other hand, has been actively studied and demonstrating a large capture range, but the accuracy is inferior to conventional optimization-based methods. We propose a fully automated pipeline using a cost function that seamlessly incorporates the reprojection error at the landmarks in CT and x-ray detected by off-the-shelf CNNs into the conventional image similarity cost, combined with the automated bone segmentation. We experimentally demonstrated that the pipeline allowed a robust and accurate 2D-3D registration to track all 12 tarsal bones, including the metatarsals at the foot arch, which is especially important in the foot biomechanics but has been unmeasurable with previous methods. We evaluated the proposed fully automated pipeline in studies using a bone phantom and real x-ray images of human subjects. The real image study showed the registration error of 0.38 ± 1.95 mm in translation and 0.38 ± 1.20\(^\circ \) in rotation for the proximal tarsal bones.
S. Mizoe and Y. Otake—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Source code was obtained from https://github.com/yuta-hi/bayesian_unet.
References
Stanković, K., Booth, B.G., Danckaers, F., Burg, F., Vermaelen, P., Duerinck, S., et al.: Three-dimensional quantitative analysis of healthy foot shape: a proof of concept study. J. Foot Ankle Res. 11(1), 8 (2018)
Nozaki, S., Watanabe, K., Kamiya, T., Katayose, M., Ogihara, N.: Three-dimensional morphological variations of the human calcaneus investigated using geometric morphometrics. Clin. Anat. 33(5), 751–758 (2020)
Eichelberger, P., Blasimann, A., Lutz, N., Krause, F., Baur, H.: A minimal markerset for three-dimensional foot function assessment: measuring navicular drop and drift under dynamic conditions. J. Foot Ankle Res. 11(1), 15 (2018)
Kim, T., Park, J.C.: Short-term effects of sports taping on navicular height, navicular drop and peak plantar pressure in healthy elite athletes: a within-subject comparison. Med. (United States) 96(46), 3–8 (2017)
Behling, A.V., Manz, S., von Tscharner, V., Nigg, B.M.: Pronation or foot movement - what is important. J. Sci. Med. Sports 23(4), 366–371 (2020)
Cao, S., et al.: In vivo kinematics of functional ankle instability patients during the stance phase of walking. Gait Posture 73, 262–268 (2019)
Lenz, A.L., et al.: Compensatory motion of the subtalar joint following tibiotalar arthrodesis. J. Bone Joint Surg. 102(7), 600–608 (2020)
Esteban, J., Grimm, M., Unberath, M., Zahnd, G., Navab, N.: Towards fully automatic X-Ray to CT registration. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 631–639. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_70
Grupp, R.B., et al.: Automatic annotation of hip anatomy in fluoroscopy for robust and efficient 2D/3D registration. Int. J. Comput. Assist. Radiol. Surg. 15(5), 759–769 (2020)
Gao, C., et al.: Generalizing spatial transformers to projective geometry with applications to 2D/3D registration. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12263, pp. 329–339. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59716-0_32
Miao, S., et al.: Dilated FCN for multi-agent 2D/3D medical image registration. In: 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, pp. 4694–4701 (2018)
Otake, Y., et al.: Intraoperative image-based multiview 2D/3D registration for image-guided orthopaedic surgery: incorporation of fiducial-based C-arm tracking and GPU-acceleration. IEEE Trans. Med. Imaging 31(4), 948–962 (2012)
Mathis, A., et al.: DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci. 21(9), 1281–1289 (2018)
Hiasa, Y., Otake, Y., Takao, M., Ogawa, T., Sugano, N., Sato, Y.: Automated muscle segmentation from clinical CT using Bayesian U-Net for personalized musculoskeletal modeling. IEEE Trans. Med. Imaging 39(4), 1030–1040 (2019)
Nikolaus, H.: The CMA evolution strategy: a comparing review. In: Lozano, J., et al. (ed.) Towards a New Evolutionary Computation, vol. 192, pp. 75–102. Springer, Heidelberg (2006). https://doi.org/10.1007/3-540-32494-1_4
Penney, G.P., et al.: A comparison of similarity measures for use in 2-D-3-D medical image registration. IEEE Trans. Med. Imaging 17(4), 586–595 (1998)
Payer, C., Štern, D., Bischof, H., Urschler, M.: Integrating spatial configuration into heatmap regression based CNNs for landmark localization. Med. Image Anal. 54, 207–219 (2019)
Jiang, H., Sun, D., Jampani, V., Yang, M.-H., Learned-Miller, E., Kautz, J.: Super SloMo: high quality estimation of multiple intermediate frames for video interpolation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9000–9008 (2018)
Bertalmio, M., Bertozzi, A.L., Sapiro, G.: Navier-stokes, fluid dynamics, and image and video inpainting. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, p. I (2001)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Mizoe, S. et al. (2021). 4D-Foot: A Fully Automated Pipeline of Four-Dimensional Analysis of the Foot Bones Using Bi-plane X-Ray Video and CT. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12904. Springer, Cham. https://doi.org/10.1007/978-3-030-87202-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-87202-1_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87201-4
Online ISBN: 978-3-030-87202-1
eBook Packages: Computer ScienceComputer Science (R0)
