Multi-task Shape Regression for Medical Image Segmentation

  • Xiantong ZhenEmail author
  • Yilong Yin
  • Mousumi Bhaduri
  • Ilanit Ben Nachum
  • David Laidley
  • Shuo Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9902)


In this paper, we propose a general segmentation framework of Multi-Task Shape Regression (MTSR) which formulates segmentation as multi-task learning to leverage its strength of jointly solving multiple tasks enhanced by capturing task correlations. The MTSR entirely estimates coordinates of all points on shape contours by multi-task regression, where estimation of each coordinate corresponds to a regression task; the MTSR can jointly handle nonlinear relationships between image appearance and shapes while capturing holistic shape information by encoding coordinate correlations, which enables estimation of highly variable shapes, even with vague edge or region inhomogeneity. The MTSR achieves a long-desired general framework without relying on any specific assumptions or initialization, which enables flexible and fully automatic segmentation of multiple objects simultaneously, for different applications irrespective of modalities. The MTSR is validated on six representative applications of diverse images, achieves consistently high performance with dice similarity coefficient (DSC) up to 0.93 and largely outperforms state of the arts in each application, which demonstrates its effectiveness and generality for medical image segmentation.



This work was supported by the NSFC Joint Fund with Guangdong under Key Project (Grant No. U1201258) and NSFC (Grant No. 61571147).


  1. 1.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. TPAMI 23(6), 681–685 (2001)CrossRefGoogle Scholar
  2. 2.
    Wang, Z., Zhen, X., Tay, K., Osman, S., Romano, W., Li, S.: Regression segmentation for \(M^3\) spinal images. TMI 34(8), 1640–1648 (2015)Google Scholar
  3. 3.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, vol. 1, pp. 886–893 (2005)Google Scholar
  4. 4.
    Nie, F., Huang, H., Cai, X., Ding, C.H.: Efficient, robust feature selection via joint \(\ell _{2,1}\)-norms minimization. In: NIPS, pp. 1813–1821 (2010)Google Scholar
  5. 5.
    Kimeldorf, G.S., Wahba, G.: A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann. Math. Stat. 41(2), 495–502 (1970)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Zhen, X., Islam, A., Bhaduri, M., Chan, I., Li, S.: Direct and simultaneous four-chamber volume estimation by multi-output regression. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 669–676. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24553-9_82CrossRefGoogle Scholar
  7. 7.
    Heimann, T., Morrison, B.J., Styner, M.A., Niethammer, M., Warfield, S.: Segmentation of knee images: a grand challenge. In: Proceedings of MICCAI Workshop on Medical Image Analysis for the Clinic, pp. 207–214 (2010)Google Scholar
  8. 8.
    Hogeweg, L., Sánchez, C.I., de Jong, P.A., Maduskar, P., van Ginneken, B.: Clavicle segmentation in chest radiographs. Med. Image Anal. 16(8), 1490–1502 (2012)CrossRefGoogle Scholar
  9. 9.
    Litjens, G., Toth, R., van de Ven, W., Hoeks, C., Kerkstra, S., van Ginneken, B., Vincent, G., Guillard, G., Birbeck, N., Zhang, J., et al.: Evaluation of prostate segmentation algorithms for MRI: the promise12 challenge. Med. Image Anal. 18(2), 359–373 (2014)CrossRefGoogle Scholar
  10. 10.
    Zhen, X., Wang, Z., Islam, A., Bhaduri, M., Chan, I., Li, S.: Multi-scale deep networks and regression forests for direct bi-ventricular volume estimation. Med. Image Anal. 30, 120–129 (2016)CrossRefGoogle Scholar
  11. 11.
    Shan, L., Zach, C., Charles, C., Niethammer, M.: Automatic atlas-based three-label cartilage segmentation from MR knee images. Med. Image Anal. 18(7), 1233–1246 (2014)CrossRefGoogle Scholar
  12. 12.
    Hara, K., Chellappa, R.: Growing regression forests by classification: applications to object pose estimation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part II. LNCS, vol. 8690, pp. 552–567. Springer, Heidelberg (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • Xiantong Zhen
    • 1
    • 2
    Email author
  • Yilong Yin
    • 3
  • Mousumi Bhaduri
    • 4
  • Ilanit Ben Nachum
    • 1
    • 2
  • David Laidley
    • 4
  • Shuo Li
    • 1
    • 2
  1. 1.Digital Imaging Group (DIG)LondonCanada
  2. 2.The University of Western OntarioLondonCanada
  3. 3.Shandong UniversityShandongChina
  4. 4.London Health Sciences CentreLondonCanada

Personalised recommendations