Shape, Appearance and Spatial Relationships

Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)


Object detection in medical image analysis can be modelled as a search for an object model in the image. The model describes attributes such as shape and appearance of the object. The search consists of fitting instances of the model to the data. A quality-of-fit measure determines whether one or several objects have been found. Generating the model for a structure of interest can be difficult. It has to include knowledge about acceptable variation of attributes within an object class while remaining suitably discriminative. Several techniques to generate and use object models will be presented in this chapter. Information about acceptable object variation in these models is either generated from training or is introduced via modeling. In either case, an efficient representation is needed with (relatively) few parameters yet being able to represent variation of shape and appearance between subjects. Applying shape (and appearance) models to the data may produce the object segmentation or they may be used as additional constraint in a subsequent segmentation process. This chapter closes with a discussion on how shape models can be used to augment data-driven segmentation by a shape model component.


Shape Variation Shape Model Medial Axis Model Instance Object Instance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Ali AM, Farag AA, El-Baz AS (2007) Graph cuts framework for kidney segmentation with prior shape constraints. In: MICCAI 2007, Part I. LNCS, vol 4791, pp 384-392Google Scholar
  2. Al-Zubi S, Toennies KD (2003) Generalizing the active shape model by integrating structural knowledge to recognize hand drawn sketches. In: Proceedings of CAIP 2003. LNCS, vol 2756, pp 320–328Google Scholar
  3. Al-Zubi S, Brömme A, Toennies K (2003) Using an active shape structural model for biometric sketch recognition. In: Joint Pattern Recognition Symposium, pp 187–195Google Scholar
  4. Artaechevarria X, Munoz-Barrutia A, Ortiz-de-Solórzano C (2009) Combination strategies in multi-atlas image segmentation: application to brain MR data. IEEE Trans Med Imaging 28(8):1266–1277CrossRefGoogle Scholar
  5. Bardinet E, Cohen LD, Ayache N (1995) Tracking medical 3D data with a parametric deformable model. In: Proceedings of IEEE international symposium computer vision, pp 299–304Google Scholar
  6. Barr AH (1992). Rigid physically based superquadrics. In: Kirk D (ed) Graphics gems III. Academic Press, pp 137–159Google Scholar
  7. Bergner S, Al-Zubi S, Toennies KD (2004) Deformable structural models. In: Proceedings of the IEEE international conference image processing ICIP, pp 1875–1878Google Scholar
  8. Biederman I (1985) Human image understanding: recent research and a theory. Comput Vis Graph Image Process 32:29–73CrossRefGoogle Scholar
  9. Binford T (1987) Generalized cylinder representation encyclopedia of artificial intelligence. Wiley, New York, pp 321–323Google Scholar
  10. Blum H (1967) A transformation for extracting new descriptors of shape. In: Proceedings of a symposium models for the perception of speech and visual form, pp 362–380Google Scholar
  11. Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23(11):1222–1239CrossRefGoogle Scholar
  12. Brett AD, Taylor CJ (1999) A framework for automated landmark generation for automated 3d statistical model construction. In: Proceedings of 16th international conference on information processing in medical imaging IPMI’99. LNCS, vol 1613, pp 376–381Google Scholar
  13. Byers R, Xu H (2008) A new scaling for Newton’s iteration for the polar decomposition and its backward stability. SIAM J Matrix Anal Appl 30(2):822–843MathSciNetCrossRefMATHGoogle Scholar
  14. Chan TF, Vese LA (2001) Active contours without edges. IEEE Trans Image Process 10(2):266–277CrossRefMATHGoogle Scholar
  15. Chen X, Udupa JK, Alavi A, Torigan DA (2013) GC-ASM: synergistic integration of graph cut and active shape model strategies for medical image segmentation. Comput Vis Image Underst 117:513–524CrossRefGoogle Scholar
  16. Cheung KW, Yeung DY, Chin RT (2002) On deformable models for visual pattern recognition. Pattern Recognit 35(7):1507–1526CrossRefMATHGoogle Scholar
  17. Chevalier L, Jaillet F, Baskurt A (2001) 3D shape coding with superquadrics. In: Proceedings of IEEE international conference image processing ICIP, II, pp 93–96Google Scholar
  18. Choi MG, Hyeong-Seok K (2005) Modal warping: real-time simulation of large rotational deformation and manipulation. IEEE Trans Vis Comput Graph 11(1):91–101CrossRefGoogle Scholar
  19. Cootes TF, Taylor CJ (1992) Active shape models—‘smart snakes’. In: Proceedings of British machine vision conferenceGoogle Scholar
  20. Cootes TF, Taylor CJ (1995) Combining point distribution models with shape models based on finite-element analysis. Image Vis Comput 13(5):403–409CrossRefGoogle Scholar
  21. Cootes TF, Taylor CJ (1999) A mixture model for representing shape variation. Image Vis Comput 17(8):567–573CrossRefGoogle Scholar
  22. Cootes TF, Hill A, Taylor CJ, Haslam J (1994) The use of active shape models for locating structures in medical images. Image Vis Comput 12(6):355–366CrossRefGoogle Scholar
  23. Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models—their training and application. Comput Vis Image Underst 61(1):38–59CrossRefGoogle Scholar
  24. Cootes TF, Edwards GJ, Taylor CJ (1998) Active appearance models. In: 5th European conference on computer vision ECCV1998. LNCS, vol 1407, pp 484–498Google Scholar
  25. Cremers D, Rousson M (2007) Efficient kernel density estimation of shape and intensity priors for level set segmentation. In: Deformable models. Springer, New York, pp 447–460Google Scholar
  26. Cuadra MB, Duay V, Thiran JP (2015) Atlas-based segmentation. In: Handbook of biomedical imaging. Springer, New York, pp 221–244Google Scholar
  27. Davies ER (1988) A modified Hough scheme for general circle location. Pattern Recognit 7(1):37–43CrossRefGoogle Scholar
  28. Delingette H, Hebert M, Ikeuchi K (1992) Shape representation and image segmentation using deformable surfaces. Image Vis Comput 10(3):132–145CrossRefGoogle Scholar
  29. Dornheim L, Toennies KD, Dornheim J (2005) Stable dynamic 3d shape models. In: IEEE international conference on image processing ICIP, III, pp 1276–1279Google Scholar
  30. Duan Z, Liang S, Bao H, Zhu S, Wang G, Zhang JJ, Chen HLu (2010) A coupled level set framework for bladder wall segmentation with application to MR cystography. IEEE Trans Med Imaging 29(3):903–915CrossRefGoogle Scholar
  31. Edelman S (1997) Computational theories in object recognition. Trends Cognit Sci 1:296–304CrossRefGoogle Scholar
  32. Engel K, Toennies KD (2008) Segmentation of the midbrain in transcranial sonographies using a two-component deformable model. In: 12th annual conference medical image understanding and analysis, pp 3–7Google Scholar
  33. Engel K, Toennies KD (2009) Hierarchical vibrations: a structural decomposition approach for image analysis. In: Energy minimization methods in computer vision and pattern recognition. LNCS, vol 5681, pp 317–330Google Scholar
  34. Engel K, Toennies KD (2010) Hierarchical vibrations for part-based recognition of complex objects. Pattern Recognit 43(8):2681–2691CrossRefMATHGoogle Scholar
  35. Engel K, Toennies KD, Brechmann A (2011) Part-based localisation and segmentation of landmark-related auditory cortical regions. Pattern Recognit 44(9):2017–2033CrossRefGoogle Scholar
  36. Farzinfar M, Xue Z, Teoh EK (2008) Joint parametric and non-parametric curve evolution for medical image segmentation. In: Europe conference computer vision (ECCV 2008), pp 167–178Google Scholar
  37. Ferrant M, Macq B, Nabavi A, Warfield SK (2000) Deformable modeling for characterizing biomedical shape changes. In: 9th international conference discrete geometry for computer imagery DGCI 2000. LNCS, vol 1953, pp 235–248Google Scholar
  38. Frangi AF, Rueckert D, Schnabel J, Niessen WJ (2001) Automatic 3d ASM construction via atlas-based landmarking and volumetric elastic registration. In: Proceedings of 17th international conference information processing in medical imaging IPMI 2001. LNCS, vol 2082, pp 78–91Google Scholar
  39. Freedman D, Zhang T (2005) Interactive graph cut based segmentation with shape priors. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2005), vol 1, pp 755–762Google Scholar
  40. Giblin P, Kimia BB (2004) A formal classification of 3d medial axis points and their local geometry. IEEE Trans Pattern Recognit Mach Intell 26(2):238–251CrossRefGoogle Scholar
  41. Gloger O, Toennies KD, Mensel B, Völzke H (2015) Fully automatized renal parenchyma volumetry using a support vector machine based recognition system for subject-specific probability map generation in native MR volume data. Phys Med Biol 60(22):8675CrossRefGoogle Scholar
  42. Gong L, Pathak SD, Haynor DR, Cho PS, Kim Y (2004) Parametric shape modeling using deformable superellipses for prostate segmentation. IEEE Trans Med Imaging 23(3):340–349CrossRefGoogle Scholar
  43. Hamarneh G, McInerney T, Terzopoulos D (2001) Deformable organisms for automatic medical image analysis. In: Medical image computing and computer-assisted intervention MICCAI 2001. LNCS, vol 2208, pp 66–76Google Scholar
  44. Heimann T, Wolf I, Meinzer HP (2006) Active shape models for a fully automated 3d segmentation of the liver—an evaluation on clinical data. In: Medical image computing and computer-assisted intervention MICCAI 2006. LNCS, vol 4191, pp 41–48Google Scholar
  45. Hu S, Coupé P, Pruessner JS, Collins DL (2011) Appearance-based modeling for segmentation of hippocampus and amygdala using multi-contrast MR imaging. Neuroimage 58(2):549–559CrossRefGoogle Scholar
  46. Iglesias JE, Sabuncu MR (2015) Multi-atlas segmentation of biomedical images: a survey. Med Image Anal 24(1):205–219CrossRefGoogle Scholar
  47. Jackway PT, Deriche M (1996) Scale-space properties of the multiscale morphological dilation-erosion. IEEE Trans Pattern Anal Mach Intell 18(1):38–51CrossRefGoogle Scholar
  48. Joshi S, Pizer SM, Fletcher PT, Yushkevich P, Thall A, Marron JS (2002) Multiscale deformable model segmentation and statistical shape analysis using medial descriptions. IEEE Trans Med Imaging 21(5):538–550CrossRefGoogle Scholar
  49. Kassim AA, Tan T, Tan KH (1999) A comparative study of efficient generalized Hough transform techniques. Image Vis Comput 17(10):737–748CrossRefGoogle Scholar
  50. Kelemen A, Székely G, Gerig G (1999) Elastic model-based segmentation of 3-D neuroradiological data sets. IEEE Trans Med Imaging 18(10):828–839CrossRefGoogle Scholar
  51. Kichenassamy S, Kumar A, Olver P, Tannenbaum A, Yezzi A (1995) Gradient flows and geometric active contour models. In: 5th international conference computer vision (ICCV’95), pp 810–817Google Scholar
  52. Kohlberger T, Uzubas MG, Alvino C, Kadir T, Slosman D, Funka-Lea G (2009) Organ segmentation with level sets using local shape and appearance priors. Medical image computing and computer-assisted intervention–MICCAI 2009. Springer, Berlin, pp 34–42CrossRefGoogle Scholar
  53. Lam L, Lee SW, Suen CY (1992) Thinning methodologies—a comprehensive survey. IEEE Trans Pattern Anal Mach Intell 14(9):869–885CrossRefGoogle Scholar
  54. Leventon ME, Grimson WEL, Faugeras O. (2000) Statistical shape influence in geodesic active contours. In: IEEE conference computer vision and pattern recognition (CVPR 2000), vol 1, pp 316–323Google Scholar
  55. Li K, Wu X, Chen DZ, Sonka M (2006) Optimal surface segmentation in volumetric images—a graph-theoretic approach. IEEE Trans PAMI 28(1):119–134Google Scholar
  56. Li K, Wu X, Chen DZ, Sonka M (2004) Efficient optimal surface detection: theory, implementation and experimental validation. In: Proceedings of SPIE international symposium medical imaging: image processing, pp 620–627Google Scholar
  57. Li X, Chen X, Yao J, Zhang X, Yang F, Tian J (2012) Automatic renal cortex segmentation using implicit shape registration and novel multiple surfaces graph search. IEEE Trans Med Imaging 31(10):1849–1860CrossRefGoogle Scholar
  58. Lim PH, Bagci U, Bai L (2013) Introducing Willmore flow into level set segmentation of spinal vertebrae. IEEE Trans Biomed Eng 60(1):115–122CrossRefGoogle Scholar
  59. Lindeberg T (1994) Scale-space theory: a basic tool for analysing structures at different scales. J Appl Stat 21(2):225–270CrossRefGoogle Scholar
  60. Liu X, Chen DZ, Tawhai MH, Wu X, Hoffman EA, Sonka M (2013) Optimal graph search based segmentation of airway double surfaces across bifurcations. IEEE Trans Med Imaging 32(3):493–510CrossRefGoogle Scholar
  61. Mandal C, Vemuri BC, Qin H (1998) A new dynamic FEM-based subdivision surface model for shape recovery and tracking in medical images. In: Medical image computing and computer-assisted intervention MICCAI’98. LNCS, vol 1496, pp 753–760Google Scholar
  62. Marr D (1983) Vision. Henry Holt & CompanyGoogle Scholar
  63. Mazziotta JC, Toga AW, Evans A, Fox P, Lancaster J (1995) A probabilistic atlas of the human brain: theory and rationale for its development: the international consortium for brain mapping (ICBM). Neuroimage 2(2):89–101CrossRefGoogle Scholar
  64. Mokhtarian F, Mackworth A (1986) Scale-based description and recognition of planar curves and two-dimensional objects. IEEE Trans Pattern Anal Mach Intell 8(1):34–43CrossRefGoogle Scholar
  65. Müller M, Gross M (2004) Interactive virtual materials. In: Proceedings of graphics interface GI’04, pp 239–246Google Scholar
  66. Nakagomi K, Shimizu A, Kobatake H, Yakami M, Fujimoto K (2013) Multi-shape graph cuts with neighbor prior constraints and its application to lunge segmentation from chest CT volume. Med Image Anal 17:62–77CrossRefGoogle Scholar
  67. Okada T, Shimada R, Sato Y, Hori M, Yokota K, Nakamoto M, Chen YW, Nakamura H, Tamura S (2007) Automated segmentation of the liver from 3d CT images using probabilistic atlas and multi-level statistical shape model. In: medical image computing and computer-assisted intervention MICCAI 2007. LNCS, vol 4791, pp 86–93Google Scholar
  68. Paloc C, Bello F, Kitney R, Darzi A (2002) Online multiresolution volumetric mass spring model for real time soft tissue deformation. In: Proceedings of 5th international conference medical image computing and computer-assisted intervention MICCAI 2002. LNCS, vol 2489, pp 219–226Google Scholar
  69. Park H, Bland PH, Meyer CR (2003) Construction of an abdominal probabilistic atlas and its application in segmentation. IEEE Trans Med Imaging 22(4):483–492CrossRefGoogle Scholar
  70. Pentland AP, Sclaroff S (1991) Closed-form solutions for physically-based modeling and reconstruction. IEEE Trans Pattern Anal Mach Intell 13(7):715–729CrossRefGoogle Scholar
  71. Petyt M (1998) Introduction to finite element vibration analysis. Cambridge University PressGoogle Scholar
  72. Pizer SM, Oliver WR, Bloomberg SH (1987) Hierarchical shape description via the multiresolution symmetric axis transforms. IEEE Trans Pattern Anal Mach Intell 9(4):505–511CrossRefGoogle Scholar
  73. Pizer SM, Fritsch DS, Yushkevich PA, Johnson VE, Chaney EL (1999) Segmentation, registration, and measurement of shape variation via image object shape. IEEE Trans Med Imaging 18(10):851–865CrossRefGoogle Scholar
  74. Provot X (1995) Deformation constraints in a mass model to describe rigid cloth behavior. In: Graph Interface, pp 147–154Google Scholar
  75. Rak M, Toennies KD (2016a) On computerized methods for spine analysis in MRI: a systematic review. Intl J Comput Assist Radiol Surg 11(8):1445–1465CrossRefGoogle Scholar
  76. Rak M, Toennies KD (2016b) A learning-free approach to whole spine vertebra localization in MRI. In: Medical image computing and computer-assisted intervention MICCAI 2016Google Scholar
  77. Rak M, Engel K, Toennies KD (2013) Closed-form hierarchical finite element models for part-based object detection. In: Vision, Modelling, and Visualization VMV 2013, pp 137–144Google Scholar
  78. Riesenhuber M, Poggio T (2000) Models of object recognition. Nat Neurosci Suppl 3:1190–1204CrossRefGoogle Scholar
  79. Rink K, Toennies KD (2007) A level set bridging force for the segmentation of dendritic spines. Computer analysis of images and patterns CAIP 2007. Springer, Berlin, pp 571–578CrossRefGoogle Scholar
  80. Rivlin E, Dickinson SJ, Rosenfeld A (1995) Recognition by functional parts. Comput Vis Image Underst 62(2):164–176CrossRefMATHGoogle Scholar
  81. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRefGoogle Scholar
  82. Sclaroff S, Pentland AP (1995) Modal matching for correspondence and recognition. IEEE Trans Pattern Anal Mach Intell 17(6):545–561CrossRefGoogle Scholar
  83. Sokoll S, Rink K, Toennies KD, Brechmann A (2008) Dynamic segmentation of the cerebral cortex in MR data using implicit active contours. In: 12th annual conference medical image understanding and analysis MIUA 2008, pp 184–188Google Scholar
  84. Song Z, Tutison N, Avants B, Gee BC (2006) Integrated graph cuts for brain MRI segmentation. Proceedings of the international conference on medical image computing and computer-assisted intervention MICCAI 2006, LNCS 4191, pp 831–838Google Scholar
  85. Taheri S, Ong SH, Chong VFH (2010) Level-set segmentation of brain tumors using a threshold-based speed function. Image Vis Comput 28(1):26–37CrossRefGoogle Scholar
  86. Terzopoulos D, Fleischer K (1988) Deformable models. Vis Comput 4(6):306–331CrossRefGoogle Scholar
  87. Terzopoulos D, Metaxas D (1991) Dynamic 3D models with local and global deformations: deformable superquadrics. IEEE Trans Pattern Anal Mach Intell 13(7):703–714CrossRefGoogle Scholar
  88. Terzopoulos D, Platt J, Barr A, Fleischer K (1987) Elastically deformable models. Proc SIGGRAPH Comput Graph 21(4):205–214Google Scholar
  89. Thompson PM, Toga AW (1997) Detection, visualization and animation of abnormal anatomic structure with a deformable probabilistic brain atlas based on random vector field transformations. Med Image Anal 1(4):271–294CrossRefGoogle Scholar
  90. Toennies KD, Rak M, Engel K (2014) Deformable part models for object detection in medical images. Biomed Eng Online 13(1):1CrossRefGoogle Scholar
  91. Toussaint GT (1978) The use of context in pattern recognition. Pattern Recognit 10(3):189–204MathSciNetCrossRefMATHGoogle Scholar
  92. Vu N, Manjunath BS (2008) Shape prior segmentation of multiple objects with graph cuts. In: IEEE computer society conference computer vision pattern recognition (CVPR 2008), pp 1–8Google Scholar
  93. Wang H, Suh JW, Das SR, Pluta JB, Craige C, Yushkevich PA (2013) Multi-atlas segmentation with joint label fusion. IEEE Trans Pattern Anal Mach Intell 35(3):611–623CrossRefGoogle Scholar
  94. Wang L, Shi F, Li G, Gao Y, Lin W, Gilmore JH, Shen D (2014) Segmentation of neonatal brain MR images using patch-driven level sets. Neuroimage 84:141–158CrossRefGoogle Scholar
  95. Xia Y, Chandra SS, Engstrom C, Strudwick MW, Crozier S, Fripp J (2014) Automatic hip cartilage segmentation from 3D MR images using arc-weighted graph searching. Phys Med Biol 59(23):7245Google Scholar
  96. Xu C, Prince JL (1998) Snakes, shapes, and gradient vector flow. IEEE Trans Image Process 7(3):359–369MathSciNetCrossRefMATHGoogle Scholar
  97. Yazdanpanah A, Hamarneh G, Smith BR, Sarunic MV (2011) Segmentation of intra-retinal layers from optical coherence tomography images using an active contour approach. IEEE Trans Med Imaging 30(2):484–496CrossRefGoogle Scholar
  98. Yedidia JS, Freeman WT, Weiss Y (2003) Understanding belief propagation and its generalizations. Explor Artif Intell New Millenn 8:236–239Google Scholar
  99. Yin Y, Zhang X, Williams R, Wu X, Anderson DD, Sonka M (2010) LOGISMOS—layered optimal graph image segmentation of multiple objects and surfaces: cartilage segmentation in the knee joint. IEEE Trans Med Imaging 29(12):2023–2037CrossRefGoogle Scholar
  100. Zeng X, Staib LH, Schultz RT, Duncan JS (1999) Segmentation and measurements of the cortex from 3-d MR images using coupled-surfaces propagation. IEEE Trans Med Imaging 18(10):927–937CrossRefGoogle Scholar
  101. Zhang S, Zhan Y, Metaxas DN (2012) Deformable segmentation via sparse representation and dictionary learning. Med Image Anal 16(7):1385–1396CrossRefGoogle Scholar
  102. Zienkiewics OC, Taylor RL, Zhu JZ (2005) The finite element method: its basis & fundamentals, 6th edn. ElsevierGoogle Scholar

Copyright information

© Springer-Verlag London Ltd. 2017

Authors and Affiliations

  1. 1.Computer Science Department, ISGOtto-von-Guericke-Universität MagdeburgMagdeburgGermany

Personalised recommendations