Abstract
Object detection in medical image analysis can be modelled as a search for an object model in the image. The model describes attributes such as shape and appearance of the object. The search consists of fitting instances of the model to the data. A quality-of-fit measure determines whether one or several objects have been found. Generating the model for a structure of interest can be difficult. It has to include knowledge about acceptable variation of attributes within an object class while remaining suitably discriminative. Several techniques to generate and use object models will be presented in this chapter. Information about acceptable object variation in these models is either generated from training or is introduced via modeling. In either case, an efficient representation is needed with (relatively) few parameters yet being able to represent variation of shape and appearance between subjects. Applying shape (and appearance) models to the data may produce the object segmentation or they may be used as additional constraint in a subsequent segmentation process. This chapter closes with a discussion on how shape models can be used to augment data-driven segmentation by a shape model component.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The notation is the usual shorthand notation for HOM operators combining the two structuring elements in a single operator. The ‘0’s represent the erosion structuring element that is applied to the inverted binary image and the ‘1’s represent the erosion structuring element that is applied to the original binary image.
- 2.
A rule of thumb borrowed from statistical pattern recognition for estimating likelihood functions from samples predict for a 100-dimensional feature space that at least 2100 samples would be needed.
- 3.
Each of the models presented here can be essentially extended such that it can replace any of the other models. However, each of the models has been built with some idea about the objects to be described and the way necessary knowledge is to be gathered. It works most efficiently if objects or scenes to which it is applied follow this idea.
- 4.
A FEM may be defined in a similar fashion than the mass-spring model by letting 1d springs being the elements. In such case, a bounded 2d or 3d object may be represented by a dense mesh of springs restricting shape variation.
- 5.
Node values and force values of all nodes of an element (and later of the complete FEM mesh) are combined in a single vector. Hence, values for a node with index i in 2-d have indices 2i and 2i + 1 in the displacement vector u and the external force vector f.
- 6.
This kind of assemblage becomes costly for the sparse matrix K if N is large. Faster methods to carry out this operation exist but the operation itself stays the same. For application in medical image analysis, however, the size of N is usually small (i.e., N ≪ 100.000) and model creation does not happen often.
- 7.
A dynamic system can be modeled without damping but damping prevents oscillation.
- 8.
These modes are not contained in an ASM, since influence from rotation and translation is removed during normalization of the training data.
References
Ali AM, Farag AA, El-Baz AS (2007) Graph cuts framework for kidney segmentation with prior shape constraints. In: MICCAI 2007, Part I. LNCS, vol 4791, pp 384-392
Al-Zubi S, Toennies KD (2003) Generalizing the active shape model by integrating structural knowledge to recognize hand drawn sketches. In: Proceedings of CAIP 2003. LNCS, vol 2756, pp 320–328
Al-Zubi S, Brömme A, Toennies K (2003) Using an active shape structural model for biometric sketch recognition. In: Joint Pattern Recognition Symposium, pp 187–195
Artaechevarria X, Munoz-Barrutia A, Ortiz-de-Solórzano C (2009) Combination strategies in multi-atlas image segmentation: application to brain MR data. IEEE Trans Med Imaging 28(8):1266–1277
Bardinet E, Cohen LD, Ayache N (1995) Tracking medical 3D data with a parametric deformable model. In: Proceedings of IEEE international symposium computer vision, pp 299–304
Barr AH (1992). Rigid physically based superquadrics. In: Kirk D (ed) Graphics gems III. Academic Press, pp 137–159
Bergner S, Al-Zubi S, Toennies KD (2004) Deformable structural models. In: Proceedings of the IEEE international conference image processing ICIP, pp 1875–1878
Biederman I (1985) Human image understanding: recent research and a theory. Comput Vis Graph Image Process 32:29–73
Binford T (1987) Generalized cylinder representation encyclopedia of artificial intelligence. Wiley, New York, pp 321–323
Blum H (1967) A transformation for extracting new descriptors of shape. In: Proceedings of a symposium models for the perception of speech and visual form, pp 362–380
Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23(11):1222–1239
Brett AD, Taylor CJ (1999) A framework for automated landmark generation for automated 3d statistical model construction. In: Proceedings of 16th international conference on information processing in medical imaging IPMI’99. LNCS, vol 1613, pp 376–381
Byers R, Xu H (2008) A new scaling for Newton’s iteration for the polar decomposition and its backward stability. SIAM J Matrix Anal Appl 30(2):822–843
Chan TF, Vese LA (2001) Active contours without edges. IEEE Trans Image Process 10(2):266–277
Chen X, Udupa JK, Alavi A, Torigan DA (2013) GC-ASM: synergistic integration of graph cut and active shape model strategies for medical image segmentation. Comput Vis Image Underst 117:513–524
Cheung KW, Yeung DY, Chin RT (2002) On deformable models for visual pattern recognition. Pattern Recognit 35(7):1507–1526
Chevalier L, Jaillet F, Baskurt A (2001) 3D shape coding with superquadrics. In: Proceedings of IEEE international conference image processing ICIP, II, pp 93–96
Choi MG, Hyeong-Seok K (2005) Modal warping: real-time simulation of large rotational deformation and manipulation. IEEE Trans Vis Comput Graph 11(1):91–101
Cootes TF, Taylor CJ (1992) Active shape models—‘smart snakes’. In: Proceedings of British machine vision conference
Cootes TF, Taylor CJ (1995) Combining point distribution models with shape models based on finite-element analysis. Image Vis Comput 13(5):403–409
Cootes TF, Taylor CJ (1999) A mixture model for representing shape variation. Image Vis Comput 17(8):567–573
Cootes TF, Hill A, Taylor CJ, Haslam J (1994) The use of active shape models for locating structures in medical images. Image Vis Comput 12(6):355–366
Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models—their training and application. Comput Vis Image Underst 61(1):38–59
Cootes TF, Edwards GJ, Taylor CJ (1998) Active appearance models. In: 5th European conference on computer vision ECCV1998. LNCS, vol 1407, pp 484–498
Cremers D, Rousson M (2007) Efficient kernel density estimation of shape and intensity priors for level set segmentation. In: Deformable models. Springer, New York, pp 447–460
Cuadra MB, Duay V, Thiran JP (2015) Atlas-based segmentation. In: Handbook of biomedical imaging. Springer, New York, pp 221–244
Davies ER (1988) A modified Hough scheme for general circle location. Pattern Recognit 7(1):37–43
Delingette H, Hebert M, Ikeuchi K (1992) Shape representation and image segmentation using deformable surfaces. Image Vis Comput 10(3):132–145
Dornheim L, Toennies KD, Dornheim J (2005) Stable dynamic 3d shape models. In: IEEE international conference on image processing ICIP, III, pp 1276–1279
Duan Z, Liang S, Bao H, Zhu S, Wang G, Zhang JJ, Chen HLu (2010) A coupled level set framework for bladder wall segmentation with application to MR cystography. IEEE Trans Med Imaging 29(3):903–915
Edelman S (1997) Computational theories in object recognition. Trends Cognit Sci 1:296–304
Engel K, Toennies KD (2008) Segmentation of the midbrain in transcranial sonographies using a two-component deformable model. In: 12th annual conference medical image understanding and analysis, pp 3–7
Engel K, Toennies KD (2009) Hierarchical vibrations: a structural decomposition approach for image analysis. In: Energy minimization methods in computer vision and pattern recognition. LNCS, vol 5681, pp 317–330
Engel K, Toennies KD (2010) Hierarchical vibrations for part-based recognition of complex objects. Pattern Recognit 43(8):2681–2691
Engel K, Toennies KD, Brechmann A (2011) Part-based localisation and segmentation of landmark-related auditory cortical regions. Pattern Recognit 44(9):2017–2033
Farzinfar M, Xue Z, Teoh EK (2008) Joint parametric and non-parametric curve evolution for medical image segmentation. In: Europe conference computer vision (ECCV 2008), pp 167–178
Ferrant M, Macq B, Nabavi A, Warfield SK (2000) Deformable modeling for characterizing biomedical shape changes. In: 9th international conference discrete geometry for computer imagery DGCI 2000. LNCS, vol 1953, pp 235–248
Frangi AF, Rueckert D, Schnabel J, Niessen WJ (2001) Automatic 3d ASM construction via atlas-based landmarking and volumetric elastic registration. In: Proceedings of 17th international conference information processing in medical imaging IPMI 2001. LNCS, vol 2082, pp 78–91
Freedman D, Zhang T (2005) Interactive graph cut based segmentation with shape priors. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2005), vol 1, pp 755–762
Giblin P, Kimia BB (2004) A formal classification of 3d medial axis points and their local geometry. IEEE Trans Pattern Recognit Mach Intell 26(2):238–251
Gloger O, Toennies KD, Mensel B, Völzke H (2015) Fully automatized renal parenchyma volumetry using a support vector machine based recognition system for subject-specific probability map generation in native MR volume data. Phys Med Biol 60(22):8675
Gong L, Pathak SD, Haynor DR, Cho PS, Kim Y (2004) Parametric shape modeling using deformable superellipses for prostate segmentation. IEEE Trans Med Imaging 23(3):340–349
Hamarneh G, McInerney T, Terzopoulos D (2001) Deformable organisms for automatic medical image analysis. In: Medical image computing and computer-assisted intervention MICCAI 2001. LNCS, vol 2208, pp 66–76
Heimann T, Wolf I, Meinzer HP (2006) Active shape models for a fully automated 3d segmentation of the liver—an evaluation on clinical data. In: Medical image computing and computer-assisted intervention MICCAI 2006. LNCS, vol 4191, pp 41–48
Hu S, Coupé P, Pruessner JS, Collins DL (2011) Appearance-based modeling for segmentation of hippocampus and amygdala using multi-contrast MR imaging. Neuroimage 58(2):549–559
Iglesias JE, Sabuncu MR (2015) Multi-atlas segmentation of biomedical images: a survey. Med Image Anal 24(1):205–219
Jackway PT, Deriche M (1996) Scale-space properties of the multiscale morphological dilation-erosion. IEEE Trans Pattern Anal Mach Intell 18(1):38–51
Joshi S, Pizer SM, Fletcher PT, Yushkevich P, Thall A, Marron JS (2002) Multiscale deformable model segmentation and statistical shape analysis using medial descriptions. IEEE Trans Med Imaging 21(5):538–550
Kassim AA, Tan T, Tan KH (1999) A comparative study of efficient generalized Hough transform techniques. Image Vis Comput 17(10):737–748
Kelemen A, Székely G, Gerig G (1999) Elastic model-based segmentation of 3-D neuroradiological data sets. IEEE Trans Med Imaging 18(10):828–839
Kichenassamy S, Kumar A, Olver P, Tannenbaum A, Yezzi A (1995) Gradient flows and geometric active contour models. In: 5th international conference computer vision (ICCV’95), pp 810–817
Kohlberger T, Uzubas MG, Alvino C, Kadir T, Slosman D, Funka-Lea G (2009) Organ segmentation with level sets using local shape and appearance priors. Medical image computing and computer-assisted intervention–MICCAI 2009. Springer, Berlin, pp 34–42
Lam L, Lee SW, Suen CY (1992) Thinning methodologies—a comprehensive survey. IEEE Trans Pattern Anal Mach Intell 14(9):869–885
Leventon ME, Grimson WEL, Faugeras O. (2000) Statistical shape influence in geodesic active contours. In: IEEE conference computer vision and pattern recognition (CVPR 2000), vol 1, pp 316–323
Li K, Wu X, Chen DZ, Sonka M (2006) Optimal surface segmentation in volumetric images—a graph-theoretic approach. IEEE Trans PAMI 28(1):119–134
Li K, Wu X, Chen DZ, Sonka M (2004) Efficient optimal surface detection: theory, implementation and experimental validation. In: Proceedings of SPIE international symposium medical imaging: image processing, pp 620–627
Li X, Chen X, Yao J, Zhang X, Yang F, Tian J (2012) Automatic renal cortex segmentation using implicit shape registration and novel multiple surfaces graph search. IEEE Trans Med Imaging 31(10):1849–1860
Lim PH, Bagci U, Bai L (2013) Introducing Willmore flow into level set segmentation of spinal vertebrae. IEEE Trans Biomed Eng 60(1):115–122
Lindeberg T (1994) Scale-space theory: a basic tool for analysing structures at different scales. J Appl Stat 21(2):225–270
Liu X, Chen DZ, Tawhai MH, Wu X, Hoffman EA, Sonka M (2013) Optimal graph search based segmentation of airway double surfaces across bifurcations. IEEE Trans Med Imaging 32(3):493–510
Mandal C, Vemuri BC, Qin H (1998) A new dynamic FEM-based subdivision surface model for shape recovery and tracking in medical images. In: Medical image computing and computer-assisted intervention MICCAI’98. LNCS, vol 1496, pp 753–760
Marr D (1983) Vision. Henry Holt & Company
Mazziotta JC, Toga AW, Evans A, Fox P, Lancaster J (1995) A probabilistic atlas of the human brain: theory and rationale for its development: the international consortium for brain mapping (ICBM). Neuroimage 2(2):89–101
Mokhtarian F, Mackworth A (1986) Scale-based description and recognition of planar curves and two-dimensional objects. IEEE Trans Pattern Anal Mach Intell 8(1):34–43
Müller M, Gross M (2004) Interactive virtual materials. In: Proceedings of graphics interface GI’04, pp 239–246
Nakagomi K, Shimizu A, Kobatake H, Yakami M, Fujimoto K (2013) Multi-shape graph cuts with neighbor prior constraints and its application to lunge segmentation from chest CT volume. Med Image Anal 17:62–77
Okada T, Shimada R, Sato Y, Hori M, Yokota K, Nakamoto M, Chen YW, Nakamura H, Tamura S (2007) Automated segmentation of the liver from 3d CT images using probabilistic atlas and multi-level statistical shape model. In: medical image computing and computer-assisted intervention MICCAI 2007. LNCS, vol 4791, pp 86–93
Paloc C, Bello F, Kitney R, Darzi A (2002) Online multiresolution volumetric mass spring model for real time soft tissue deformation. In: Proceedings of 5th international conference medical image computing and computer-assisted intervention MICCAI 2002. LNCS, vol 2489, pp 219–226
Park H, Bland PH, Meyer CR (2003) Construction of an abdominal probabilistic atlas and its application in segmentation. IEEE Trans Med Imaging 22(4):483–492
Pentland AP, Sclaroff S (1991) Closed-form solutions for physically-based modeling and reconstruction. IEEE Trans Pattern Anal Mach Intell 13(7):715–729
Petyt M (1998) Introduction to finite element vibration analysis. Cambridge University Press
Pizer SM, Oliver WR, Bloomberg SH (1987) Hierarchical shape description via the multiresolution symmetric axis transforms. IEEE Trans Pattern Anal Mach Intell 9(4):505–511
Pizer SM, Fritsch DS, Yushkevich PA, Johnson VE, Chaney EL (1999) Segmentation, registration, and measurement of shape variation via image object shape. IEEE Trans Med Imaging 18(10):851–865
Provot X (1995) Deformation constraints in a mass model to describe rigid cloth behavior. In: Graph Interface, pp 147–154
Rak M, Toennies KD (2016a) On computerized methods for spine analysis in MRI: a systematic review. Intl J Comput Assist Radiol Surg 11(8):1445–1465
Rak M, Toennies KD (2016b) A learning-free approach to whole spine vertebra localization in MRI. In: Medical image computing and computer-assisted intervention MICCAI 2016
Rak M, Engel K, Toennies KD (2013) Closed-form hierarchical finite element models for part-based object detection. In: Vision, Modelling, and Visualization VMV 2013, pp 137–144
Riesenhuber M, Poggio T (2000) Models of object recognition. Nat Neurosci Suppl 3:1190–1204
Rink K, Toennies KD (2007) A level set bridging force for the segmentation of dendritic spines. Computer analysis of images and patterns CAIP 2007. Springer, Berlin, pp 571–578
Rivlin E, Dickinson SJ, Rosenfeld A (1995) Recognition by functional parts. Comput Vis Image Underst 62(2):164–176
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326
Sclaroff S, Pentland AP (1995) Modal matching for correspondence and recognition. IEEE Trans Pattern Anal Mach Intell 17(6):545–561
Sokoll S, Rink K, Toennies KD, Brechmann A (2008) Dynamic segmentation of the cerebral cortex in MR data using implicit active contours. In: 12th annual conference medical image understanding and analysis MIUA 2008, pp 184–188
Song Z, Tutison N, Avants B, Gee BC (2006) Integrated graph cuts for brain MRI segmentation. Proceedings of the international conference on medical image computing and computer-assisted intervention MICCAI 2006, LNCS 4191, pp 831–838
Taheri S, Ong SH, Chong VFH (2010) Level-set segmentation of brain tumors using a threshold-based speed function. Image Vis Comput 28(1):26–37
Terzopoulos D, Fleischer K (1988) Deformable models. Vis Comput 4(6):306–331
Terzopoulos D, Metaxas D (1991) Dynamic 3D models with local and global deformations: deformable superquadrics. IEEE Trans Pattern Anal Mach Intell 13(7):703–714
Terzopoulos D, Platt J, Barr A, Fleischer K (1987) Elastically deformable models. Proc SIGGRAPH Comput Graph 21(4):205–214
Thompson PM, Toga AW (1997) Detection, visualization and animation of abnormal anatomic structure with a deformable probabilistic brain atlas based on random vector field transformations. Med Image Anal 1(4):271–294
Toennies KD, Rak M, Engel K (2014) Deformable part models for object detection in medical images. Biomed Eng Online 13(1):1
Toussaint GT (1978) The use of context in pattern recognition. Pattern Recognit 10(3):189–204
Vu N, Manjunath BS (2008) Shape prior segmentation of multiple objects with graph cuts. In: IEEE computer society conference computer vision pattern recognition (CVPR 2008), pp 1–8
Wang H, Suh JW, Das SR, Pluta JB, Craige C, Yushkevich PA (2013) Multi-atlas segmentation with joint label fusion. IEEE Trans Pattern Anal Mach Intell 35(3):611–623
Wang L, Shi F, Li G, Gao Y, Lin W, Gilmore JH, Shen D (2014) Segmentation of neonatal brain MR images using patch-driven level sets. Neuroimage 84:141–158
Xia Y, Chandra SS, Engstrom C, Strudwick MW, Crozier S, Fripp J (2014) Automatic hip cartilage segmentation from 3D MR images using arc-weighted graph searching. Phys Med Biol 59(23):7245
Xu C, Prince JL (1998) Snakes, shapes, and gradient vector flow. IEEE Trans Image Process 7(3):359–369
Yazdanpanah A, Hamarneh G, Smith BR, Sarunic MV (2011) Segmentation of intra-retinal layers from optical coherence tomography images using an active contour approach. IEEE Trans Med Imaging 30(2):484–496
Yedidia JS, Freeman WT, Weiss Y (2003) Understanding belief propagation and its generalizations. Explor Artif Intell New Millenn 8:236–239
Yin Y, Zhang X, Williams R, Wu X, Anderson DD, Sonka M (2010) LOGISMOS—layered optimal graph image segmentation of multiple objects and surfaces: cartilage segmentation in the knee joint. IEEE Trans Med Imaging 29(12):2023–2037
Zeng X, Staib LH, Schultz RT, Duncan JS (1999) Segmentation and measurements of the cortex from 3-d MR images using coupled-surfaces propagation. IEEE Trans Med Imaging 18(10):927–937
Zhang S, Zhan Y, Metaxas DN (2012) Deformable segmentation via sparse representation and dictionary learning. Med Image Anal 16(7):1385–1396
Zienkiewics OC, Taylor RL, Zhu JZ (2005) The finite element method: its basis & fundamentals, 6th edn. Elsevier
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2017 Springer-Verlag London Ltd.
About this chapter
Cite this chapter
Toennies, K.D. (2017). Shape, Appearance and Spatial Relationships. In: Guide to Medical Image Analysis. Advances in Computer Vision and Pattern Recognition. Springer, London. https://doi.org/10.1007/978-1-4471-7320-5_11
Download citation
DOI: https://doi.org/10.1007/978-1-4471-7320-5_11
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-4471-7318-2
Online ISBN: 978-1-4471-7320-5
eBook Packages: Computer ScienceComputer Science (R0)