Robust Interactive Multi-label Segmentation with an Advanced Edge Detector

  • Sabine MüllerEmail author
  • Peter Ochs
  • Joachim Weickert
  • Norbert Graf
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9796)


Recent advances on convex relaxation methods allow for a flexible formulation of many interactive multi-label segmentation methods. The building blocks are a likelihood specified for each pixel and each label, and a penalty for the boundary length of each segment. While many sophisticated likelihood estimations based on various statistical measures have been investigated, the boundary length is usually measured in a metric induced by simple image gradients. We show that complementing these methods with recent advances of edge detectors yields an immense quality improvement. A remarkable feature of the proposed method is the ability to correct some erroneous labels, when computer generated initial labels are considered. This allows us to improve state-of-the-art methods for motion segmentation in videos by 5–10 % with respect to the F-measure (Dice score).



This work was partially funded by the European Union’s FP7 under the project Computational Horizons in Cancer (grant agreement No. 600841).


  1. 1.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)CrossRefGoogle Scholar
  2. 2.
    Bauer, S., Porz, N., Meier, R., Pica, A., Slotboom, J., Wiest, R., Reyes, M.: Interactive segmentation of MR images from brain tumor patients. In: Proceedings of the 11th International Symposium on Biomedical Imaging, Beijing, China, pp. 862–865 (2014)Google Scholar
  3. 3.
    Bergbauer, J., Nieuwenhuis, C., Souiai, M., Cremers, D.: Proximity priors for variational semantic segmentation and recognition. In: ICCV Workshop on Graphical Models for Scene Understanding, Sydney, Australia, pp. 15–21 (2013)Google Scholar
  4. 4.
    Boykov, Y.Y., Jolly, M.P.: Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: Proceedings of the 2001 IEEE International Conference on Computer Vision, Vancouver, Canada, pp. 105–112 (2001)Google Scholar
  5. 5.
    Brox, T., Malik, J.: Object segmentation by long term analysis of point trajectories. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 282–295. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 679–698 (1986)CrossRefGoogle Scholar
  7. 7.
    Chambolle, A., Cremers, D., Pock, T.: A convex approach to minimal partitions. SIAM J. Appl. Math. 5(4), 1113–1158 (2012)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vision 40(1), 120–145 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)CrossRefzbMATHGoogle Scholar
  10. 10.
    Diebold, J., Demmel, N., Hazırbaş, C., Moeller, M., Cremers, D.: Interactive multi-label segmentation of RGB-D images. In: Aujol, J.-F., Nikolova, M., Papadakis, N. (eds.) SSVM 2015. LNCS, vol. 9087, pp. 294–306. Springer, Heidelberg (2015)Google Scholar
  11. 11.
    Dollár, P., Zitnick, C.L.: Structured forests for fast edge detection. In: Proceedings of the 2013 IEEE International Conference on Computer Vision, Washington, DC, USA, pp. 1841–1848 (2013)Google Scholar
  12. 12.
    Dollár, P., Zitnick, C.L.: Fast edge detection using structured forests. IEEE Trans. Pattern Anal. Mach. Intell. 37(8), 1558–1570 (2015)CrossRefGoogle Scholar
  13. 13.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vision 59(2), 167–181 (2004)CrossRefGoogle Scholar
  14. 14.
    Feng, C., Zhao, D., Huang, M.: Segmentation of stroke lesions in multi-spectral MR images using bias correction embedded FCM and three phase level set. In: MICCAI Ischemic Stroke Lesion Segmentation, p. 3 (2015)Google Scholar
  15. 15.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)CrossRefzbMATHGoogle Scholar
  16. 16.
    Keuper, M., Andres, B., Brox, T.: Motion trajectory segmentation via minimum cost multicuts. In: Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Chile, pp. 3271–3279 (2015)Google Scholar
  17. 17.
    Mumford, D., Shah, J.: Optimal approximation of piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42, 577–685 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Nieuwenhuis, C., Cremers, D.: Spatially varying color distributions for interactive multilabel segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 35(5), 1234–1247 (2013)CrossRefGoogle Scholar
  19. 19.
    Nieuwenhuis, C., Hawe, S., Kleinsteuber, M., Cremers, D.: Co-sparse textural similarity for interactive segmentation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part VI. LNCS, vol. 8694, pp. 285–301. Springer, Heidelberg (2014)Google Scholar
  20. 20.
    Ochs, P., Brox, T.: Object segmentation in video: a hierarchical variational approach for turning point trajectories into dense regions. In: Proceedings of the 2011 IEEE International Conference on Computer Vision, Barcelona, Spain, pp. 1583–1590 (2011)Google Scholar
  21. 21.
    Ochs, P., Malik, J., Brox, T.: Segmentation of moving objects by long term video analysis. IEEE Trans. Pattern Anal. Mach. Intell. 36(6), 1187–1200 (2014)CrossRefGoogle Scholar
  22. 22.
    Perona, P., Malik, J.: Scale space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639 (1990)CrossRefGoogle Scholar
  23. 23.
    Rother, C., Kolmogorov, V., Blake, A.: “GrabCut”: interactive foreground extraction using iterated graph cuts. In: Proceedings of the SIGGRAPH 2004, New York, NY, USA, pp. 309–314, August 2004Google Scholar
  24. 24.
    Santner, J., Pock, T., Bischof, H.: Interactive multi-label segmentation. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 397–410. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  25. 25.
    Santner, J., Unger, M., Pock, T., Leistner, C., Saffari, A., Bischof, H.: Interactive texture segmentation using random forests and total variation. In: Proceedings of the 2009 British Machine Vision Conference, London, UK, pp. 66.1–66.12 (2009)Google Scholar
  26. 26.
    Strekalovskiy, E., Cremers, D.: Generalized ordering constraints for multilabel optimization. In: Proceedings of the 2011 IEEE International Conference on Computer Vision, Barcelona, Spain, pp. 2619–2626 (2011)Google Scholar
  27. 27.
    Strekalovskiy, E., Nieuwenhuis, C., Cremers, D.: Nonmetric priors for continuous multilabel optimization. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VII. LNCS, vol. 7578, pp. 208–221. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  28. 28.
    Unger, M., Pock, T., Trobin, W., Cremers, D., Bischof, H.: TVSeg - interactive total variation based image segmentation. In: Proceedings of the 2008 British Machine Vision Conference, Leeds, UK, pp. 40.1–40.10 (2008)Google Scholar
  29. 29.
    Werlberger, M., Unger, M., Pock, T., Bischof, H.: Efficient minimization of the non-local potts model. In: Bruckstein, A.M., ter Haar Romeny, B.M., Bronstein, A.M., Bronstein, M.M. (eds.) SSVM 2011. LNCS, vol. 6667, pp. 314–325. Springer, Heidelberg (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Sabine Müller
    • 1
    • 2
    Email author
  • Peter Ochs
    • 2
  • Joachim Weickert
    • 2
  • Norbert Graf
    • 1
  1. 1.Department of Pediatric Oncology and HematologySaarland University HospitalHomburgGermany
  2. 2.Mathematical Image Analysis GroupSaarland UniversitySaarbrückenGermany

Personalised recommendations