Abstract
The detection and localization of single or multiple landmarks is a crucial task in medical imaging. It is often required as initialization for other tasks like segmentation or registration. A common approach to localize multiple landmarks is to exploit their spatial correlations, e.g., by using a conditional random field (CRF) to incorporate geometric information between landmark pairs. This CRF is usually applied to resolve ambiguities of a localizer, e.g., a random forest or a deep neural network. In this paper, we apply a random forest/CRF combination to the task of jointly detecting and localizing 6 landmarks in the lower extremities, taken from a dataset of 660 X-ray images. The dataset is challenging since a significant number of images does not show all the landmarks. Furthermore, 11.3% of the target landmarks are altered by prostheses or pathologies.
To account for this, we introduce a “missing” label for each landmark (represented by a node in the CRF). Moreover, instead of manually specifying the CRF model by selecting suitable potential functions and the graph topology, we suggest to automatically optimize both in a learning framework. Specifically, we define a pool of potential functions and learn their CRF weights (relative contributions), in addition to the potential values in case of missing landmarks. Potentials with a low weight are removed, thus optimizing the graph topology. Detailed evaluations on our database show the feasibility of our approach. Our algorithm removed on average 23 of the initial 51 CRF potentials, and correctly detected and localized (within 10 mm tolerance) on average 92.8% of the landmarks, with individual rates ranging from 90.0% to 97.4%.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bergtholdt, M., Kappes, J.H., Schnörr, C.: Learning of graphical models and efficient inference for object class recognition. In: Franke, K., Müller, K.-R., Nickolay, B., Schäfer, R. (eds.) DAGM 2006. LNCS, vol. 4174, pp. 273–283. Springer, Heidelberg (2006). doi:10.1007/11861898_28
Bergtholdt, M., et al.: A study of parts-based object class detection using complete graphs. IJCV 87(1), 93–117 (2010)
Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: binary robust independent elementary features. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 778–792. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15561-1_56
Criminisi, A., et al.: Regression forests for efficient anatomy detection and localization in computed tomography scans. Med. Image Anal. 17(8), 1293–1303 (2013)
Donner, R., et al.: Sparse MRF appearance models for fast anatomical structure localisation. In: BMVC (2007)
Donner, R., et al.: Global localization of 3d anatomical structures by pre-filtered hough forests and discrete optimization. Med. Image Anal. 17(8), 1304–1314 (2013)
Glocker, B., Zikic, D., Konukoglu, E., Haynor, D.R., Criminisi, A.: Vertebrae localization in pathological spine CT via dense classification from sparse annotations. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 262–270. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40763-5_33
Gooßen, A.: Computational Imaging in Orthopaedic Radiography. BoD (2012)
Hahmann, F., et al.: Model interpolation for eye localization using the discriminative generalized hough transform. In: BIOSIG (2012)
Ishikawa, H.: Higher-order clique reduction in binary graph cut. In: CVPR, pp. 2993–3000. IEEE (2009)
Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: ICLR (2014)
Komodakis, N., Xiang, B., Paragios, N.: A framework for efficient structured max-margin learning of high-order mrf models. IEEE TPAMI 37(7), 1425–1441 (2015)
LeCun, Y., Chopra, S., Hadsell, R.: A tutorial on energy-based learning. In: Predicting Structured Data (2006)
Mader, A.O., Schramm, H., Meyer, C.: Efficient epiphyses localization using regression tree ensembles and a conditional random field. Bildverarbeitung für die Medizin 2017. Informatik aktuell, pp. 179–184. Springer, Heidelberg (2017). doi:10.1007/978-3-662-54345-0_42
Payer, C., Štern, D., Bischof, H., Urschler, M.: Regressing heatmaps for multiple landmark localization using CNNs. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 230–238. Springer, Cham (2016). doi:10.1007/978-3-319-46723-8_27
Ruppertshofen, H., et al.: Discriminative generalized hough transform for localization of joints in the lower extremities. CSRD 26(1), 97–105 (2011)
Ruppertshofen, H., et al.: Shape model training for concurrent localization of the left and right knee. In: SPIE Medical Imaging (2011)
Štern, D., Ebner, T., Urschler, M.: From local to global random regression forests: exploring anatomical landmark localization. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 221–229. Springer, Cham (2016). doi:10.1007/978-3-319-46723-8_26
Wang, C., Komodakis, N., Paragios, N.: Markov random field modeling, inference & learning in computer vision & image understanding: a survey. CVIU 117, 1610–1627 (2013)
Acknowledgements
The authors thank the Diagnosezentrum Urania, Vienna and the Dartmouth Hitchcock Medical Center, Lebanon for providing the radiographs that served as training and test sets; Gooßen [8] for the annotations. This work has been financially supported by the Federal Ministry of Education and Research under the grant 03FH013IX5. The liability for the content of this work lies with the authors.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Mader, A.O. et al. (2017). Detection and Localization of Landmarks in the Lower Extremities Using an Automatically Learned Conditional Random Field. In: Cardoso, M., et al. Graphs in Biomedical Image Analysis, Computational Anatomy and Imaging Genetics. GRAIL MICGen MFCA 2017 2017 2017. Lecture Notes in Computer Science(), vol 10551. Springer, Cham. https://doi.org/10.1007/978-3-319-67675-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-67675-3_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-67674-6
Online ISBN: 978-3-319-67675-3
eBook Packages: Computer ScienceComputer Science (R0)