Toward Automated Validation of Sketch-Based 3D Segmentation Editing Tools

  • Frank Heckel
  • Momchil I. Ivanov
  • Jan H. Moltz
  • Horst K. Hahn
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7944)


Segmentation is one of the main tasks in medical image analysis. Measuring the quality of 3D segmentation algorithms is an essential requirement during development and for evaluation. Various methods exist to measure the quality of a segmentation with respect to a reference segmentation. Validating interactive 3D segmentation approaches or methods for 3D segmentation editing is more complex, however. Using interactive tools, the user plays a central role during the segmentation process as he or she needs to react on intermediate results, making established static validation approaches insufficient. In this paper we present a method to automatically generate plausible user inputs for 3D sketch-based segmentation editing algorithms, to allow an objective and reproducible validation and comparison of such tools. The user inputs are generated iteratively based on the intermediate and the reference segmentation, while static quality measurements are tracked over time. We present first results where we have compared two segmentation editing algorithms using our framework.


Validation Evaluation Interactive Segmentation Segmentation Editing Simulation Automation 


  1. 1.
    Boehm, B.W.: Verifying and validating software requirements and design specifications. IEEE Software 1(1), 75–88 (1984)CrossRefGoogle Scholar
  2. 2.
    Deng, X., Du, G.: Editorial: 3D segmentation in the clinic: A grand challenge II - liver tumor segmentation (2008),
  3. 3.
    Heckel, F., Braunewell, S., Soza, G., Tietjen, C., Hahn, H.K.: Sketch-based image-independent editing of 3D tumor segmentations using variational interpolation. In: Eurographics Workshop on Visual Computing for Biology and Medicine, pp. 73–80. Eurographics Association (2012)Google Scholar
  4. 4.
    Heckel, F., Konrad, O., Hahn, H.K., Peitgen, H.O.: Interactive 3D medical image segmentation with energy-minimizing implicit functions. Computers & Graphics: Special Issue on Visual Computing for Biology and Medicine 35(2), 275–287 (2011)Google Scholar
  5. 5.
    Heckel, F., Moltz, J.H., Bornemann, L., Dicken, V., Bauknecht, H.C., Fabel, M., Hittinger, M., Kießling, A., Meier, S., Püsken, M., Peitgen, H.O.: 3D contour based local manual correction of tumor segmentations in CT scans. In: SPIE Medical Imaging: Image Processing, vol. 7259, p. 72593L. SPIE (2009)Google Scholar
  6. 6.
    Heimann, T., van Ginneken, B., Styner, M.A., Arzhaeva, Y., Aurich, V., Bauer, C., Beck, A., Becker, C., Beichel, R., Bekes, G., Bello, F., Binnig, G., Bischof, H., Bornik, A., Cashman, P.M.M., Chi, Y., Córdova, A., Dawant, B.M., Fidrich, M., Furst, J.D., Furukawa, D., Grenacher, L., Hornegger, J., Kainmuller, D., Kitney, R.I., Kobatake, H., Lamecker, H., Lange, T., Lee, J., Lennon, B., Li, R., Li, S., Meinzer, H.P., Németh, G., Raicu, D.S., Rau, A.M., van Rikxoort, E.M., Rousson, M., Ruskó, L., Saddi, K.A., Schmidt, G., Seghers, D., Shimizu, A., Slagmolen, P., Sorantin, E., Soza, G., Susomboon, R., Waite, J.M., Wimmer, A., Wolf, I.: Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE Transactions on Medical Imaging 28(8), 1251–1265 (2009)CrossRefGoogle Scholar
  7. 7.
    McGuinness, K., O’Connor, N.E.: A comparative evaluation of interactive segmentation algorithms. Pattern Recognition 43(2), 434–444 (2010)zbMATHCrossRefGoogle Scholar
  8. 8.
    McGuinness, K., O’Connor, N.E.: Toward automated evaluation of interactive segmentation. Computer Vision and Image Understanding 115(6), 868–884 (2011)CrossRefGoogle Scholar
  9. 9.
    Moschidis, E., Graham, J.: Simulation of user interaction for performance evaluation of interactive image segmentation methods. In: Medical Image Understanding and Analysis, pp. 209–213 (2009)Google Scholar
  10. 10.
    Moschidis, E., Graham, J.: A systematic performance evaluation of interactive image segmentation methods based on simulated user interaction. In: IEEE International Symposium on Biomedical Imaging, pp. 928–931 (2010)Google Scholar
  11. 11.
    Pham, D.L., Xu, C., Prince, J.L.: Current methods in medical image segmentation. Annual Review of Biomedical Engineering 2(1), 315–337 (2000)CrossRefGoogle Scholar
  12. 12.
    Udupa, J.K., LeBlanc, V.R., Zhuge, Y., Imielinska, C., Schmidt, H., Currie, L.M., Hirsch, B.E., Woodburn, J.: A framework for evaluating image segmentation algorithms. Computerized Medical Imaging and Graphics 30(2), 75–87 (2006)CrossRefGoogle Scholar
  13. 13.
    Withey, D.J., Koles, Z.J.: Medical image segmentation: Methods and software. In: International Symposium on Noninvasive Functional Source Imaging of the Brain and Heart and the International Conference on Functional Biomedical Imaging, pp. 140–143 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Frank Heckel
    • 1
  • Momchil I. Ivanov
    • 1
    • 2
  • Jan H. Moltz
    • 1
  • Horst K. Hahn
    • 1
    • 2
  1. 1.Fraunhofer MEVISBremenGermany
  2. 2.Jacobs UniversityBremenGermany

Personalised recommendations