Abstract
The application of deep learning models to large-scale data sets requires means for automatic quality assurance. We have previously developed a fully automatic algorithm for carotid artery wall segmentation in black-blood MRI that we aim to apply to large-scale data sets. This method identifies nested artery walls in 3D patches centered on the carotid artery. In this study, we investigate to what extent the uncertainty in the model predictions for the contour location can serve as a surrogate for error detection and, consequently, automatic quality assurance. We express the quality of automatic segmentations using the Dice similarity coefficient. The uncertainty in the model’s prediction is estimated using either Monte Carlo dropout or test-time data augmentation. We found that (1) including uncertainty measurements did not degrade the quality of the segmentations, (2) uncertainty metrics provide a good proxy of the quality of our contours if the center found during the first step is enclosed in the lumen of the carotid artery and (3) they could be used to detect low-quality segmentations at the participant level. This automatic quality assurance tool might enable the application of our model in large-scale data sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alblas, D., Brune, C., Wolterink, J.M.: Deep-learning-based carotid artery vessel wall segmentation in black-blood MRI using anatomical priors. In: Medical Imaging 2022: Image Processing, vol. 12032, pp. 237–244. SPIE (2022). https://doi.org/10.1117/12.2611112
Ayhan, M.S., Kühlewein, L., Aliyeva, G., Inhoffen, W., Ziemssen, F., Berens, P.: Expert-validated estimation of diagnostic uncertainty for deep neural networks in diabetic retinopathy detection. Med. Image Anal. 64, 101724 (2020). https://doi.org/10.1016/j.media.2020.101724
Balu, N., Yarnykh, V.L., Chu, B., Wang, J., Hatsukami, T., Yuan, C.: Carotid plaque assessment using fast 3D isotropic resolution black-blood MRI. Magn. Reson. Med. 65(3), 627–637 (2011). https://doi.org/10.1002/mrm.22642
Chambless, L.E., et al.: Carotid wall thickness is predictive of incident clinical stroke: the atherosclerosis risk in communities (ARIC) study. Am. J. Epidemiol. 151(5), 478–487 (2000). https://doi.org/10.1093/oxfordjournals.aje.a010233
Gal, Y., Ghahramani, Z.: Bayesian convolutional neural networks with Bernoulli approximate variational inference (2016). Comment: 12 pages, 3 figures, ICLR format, updated with reviewer comments. https://doi.org/10.48550/arXiv.1506.02158
Huang, X., Wang, J., Li, Z.: 3D carotid artery segmentation using shape-constrained active contours. Comput. Biol. Med. 153, 106530 (2023). https://doi.org/10.1016/j.compbiomed.2022.106530
Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017, pp. 5580–5590. Curran Associates Inc., Red Hook, NY, USA (2017)
Lai, S.M., Studenski, S., Duncan, P.W., Perera, S.: Persisting consequences of stroke measured by the stroke impact scale. Stroke 33(7), 1840–1844 (2002). https://doi.org/10.1161/01.STR.0000019289.15440.F2
Lavrova, E., et al.: UR-CarA-Net: a cascaded framework with uncertainty regularization for automated segmentation of carotid arteries on black blood MR images. IEEE Access 11, 26637–26651 (2023). https://doi.org/10.1109/ACCESS.2023.3258408
Phan, T.G., et al.: Carotid artery anatomy and geometry as risk factors for carotid atherosclerotic disease. Stroke 43(6), 1596–1601 (2012). https://doi.org/10.1161/STROKEAHA.111.645499
Roy, A.G., Conjeti, S., Navab, N., Wachinger, C.: Bayesian QuickNAT: model uncertainty in deep whole-brain segmentation for structure-wise quality control. Neuroimage 195, 11–22 (2019). https://doi.org/10.1016/j.neuroimage.2019.03.042
Wafa, H.A., Wolfe, C.D.A., Emmett, E., Roth, G.A., Johnson, C.O., Wang, Y.: Burden of stroke in Europe. Stroke 51(8), 2418–2427 (2020). https://doi.org/10.1161/STROKEAHA.120.029606
Wang, G., Li, W., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T.: Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing 338, 34–45 (2019). https://doi.org/10.1016/j.neucom.2019.01.103
Wang, Y., Yao, Y.: Application of artificial intelligence methods in carotid artery segmentation: a review. IEEE Access 11, 13846–13858 (2023). https://doi.org/10.1109/ACCESS.2023.3243162
Yuan, C., et al.: Carotid vessel wall segmentation challenge (2021). https://doi.org/10.5281/zenodo.4575301
Zhao, X., Li, R., Hippe, D.S., Hatsukami, T.S., Yuan, C., CARE-II Investigators: Chinese atherosclerosis risk evaluation (CARE II) study: a novel cross-sectional, multicentre study of the prevalence of high-risk atherosclerotic carotid plaque in Chinese patients with ischaemic cerebrovascular events—design and rationale. Stroke Vasc. Neurol. 2(1), 15–20 (2017). https://doi.org/10.1136/svn-2016-000053
Ziegler, M., et al.: Automated segmentation of the individual branches of the carotid arteries in contrast-enhanced MR angiography using DeepMedic. BMC Med. Imaging 21(1), 38 (2021). https://doi.org/10.1186/s12880-021-00568-6
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Thibeau-Sutre, E., Alblas, D., Buurman, S., Brune, C., Wolterink, J.M. (2023). Uncertainty-Based Quality Assurance of Carotid Artery Wall Segmentation in Black-Blood MRI. In: Sudre, C.H., Baumgartner, C.F., Dalca, A., Mehta, R., Qin, C., Wells, W.M. (eds) Uncertainty for Safe Utilization of Machine Learning in Medical Imaging. UNSURE 2023. Lecture Notes in Computer Science, vol 14291. Springer, Cham. https://doi.org/10.1007/978-3-031-44336-7_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-44336-7_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44335-0
Online ISBN: 978-3-031-44336-7
eBook Packages: Computer ScienceComputer Science (R0)
