Skip to main content

Advertisement

SpringerLink
Log in

Point-based medialness for 2D shape description and identification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

We propose a perception-based medial point description of a natural form (2D: static or in articulated movement) as a framework for a shape representation which can then be efficiently used in biological species identification and matching tasks. Medialness is defined by adapting and refining a definition first proposed in the cognitive science literature when studying the visual attention of human subjects presented with articulated biological 2D forms in movement, such as horses, dogs and humans (walking, running). In particular, special loci of high medialness for the interior of a form in movement, referred to as “hot spots”, prove most attractive to the human perceptual system. We propose an algorithmic process to identify such hot spots. In this article we distinguish exterior from interior shape representation. We further augment hot spots with extremities of medialness ridges identifying significant concavities (from outside) and convexities (from inside). Our representation is strongly footed in results from cognitive psychology, but also inspired by know-how in art and animation, and the algorithmic part is influenced by techniques from more traditional computer vision. A robust shape matching algorithm is designed that finds the most relevant targets from a database of templates by comparing feature points in a scale, rotation and translation invariant way. The performance of our method has been tested on several databases. The robustness of the algorithm is further tested by perturbing the data-set at different levels.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28

Similar content being viewed by others

Notes

  1. A “shape context” is defined centered at each feature contour point, by seeking within a circular polar grid proximal contour neighbors and creating a descriptive histogram from orientation-labeled bins [5]. It represents a discrete approximation of relative contour similarity where a region of influence for each considered feature contour point is being evaluated.

  2. Medial-reps or M-reps, previously referred to as cores, are a discrete alternative to Blum’s medial axis, developed by Stephen Pizer and his collaborators, where medial atoms (loci) are selected in a sparse sampling of a main or long medial axis and linked to the object boundary via spokes normal to that boundary at their attachment points [40]. Such a sparse connected representation of medialness is well adapted to elongated forms, such as found in medical imaging, when modeling various body tissues [41]. This approach is model based, where a sparse connected skeletal grid is retro-fitted to the outline of an object segmented in an image (in 2D or 3D).

  3. Where the contour fragments are built alike the method of Wang et al. [55].

  4. Currently the threshold value is globally set (the typical value is 10 for the intensity range 0-255) and remains constant for the whole DB.

  5. The non-negativeness (of the scalar product \(\overrightarrow {v_{b}}\cdot \overrightarrow {v_{(b,p)}}\)) is used to rule out boundary pixels which are oriented away from the given annulus centre. We do not consider the geometry (differential continuity) of a contour other than provided by that gradient orientation. NB: this criterion is efficient if we have reliable figure-ground information. This is a limit of the modified gauge \(D_{\epsilon }^{\ast }\); however we can always fall back on the original gauge D 𝜖 if object segmentation is not reliable.

  6. The tolerance value (𝜖) is currently set as the elementary pixel size (and so is related to the resolution used).

  7. In the recent cognitive science literature, arguments are presented to support the idea that medialness can be the basis of figure-ground segregation [28].

  8. This heuristic, of positioning the representative concavity near the object contour trace is useful both for visualisation and for greater robustness in matching under articulated movements.

  9. The precise definition and study of an object part is in itself an important topic which requires a separate presentation, including its precise use in characterising articulated movement. Note also that part perception is studied in psychology where it is shown to be an important cue for human’s ability to deal with occlusions and other partial visual information [7].

  10. We use the traditional 3D graphics notation when performing affine transformation using 4×4 matrices; as we are only dealing with a 2D problem, one of the spatial dimensions is redundant, but this is not a problem in practice.

  11. Our first results on such data were presented in a short paper at the 1st International Workshop on “Environmental Multimedia Retrieval” (EMR) held in conjunction with the ACM International Conference on Multimedia Retrieval (ICMR) in Glasgow (UK), April 1, 2014 [1].

  12. We do not claim that this way of perturbing the data is physically accurate in modeling natural decay or erosion. Rather it provides a simple (computational) way to approximate such effects and produce deformations which appear (visually) credible in modeling these.

  13. The binary value of rel(r) is set automatically by parsing a text file which contains the ground truth for each image: i.e., which type it corresponds to, such as a dog, a horse, etc. Thus, the AP measure is only available given ground truth (identification) is provided.

  14. An example of such a combination of methods to exploit their relative strengths has recently been proposed by Nanni et al. who use [38]: Shape Contexts, Inner Distance, Height Functions, Shapelets, traditional Curvature, Fast Radial Symmetry Transform, Local Phase Quantization, Histogram of Gradients, Wavelets. Shape descriptors (contour based) are compared using a Weighted Spectral Distance to measure dissimilarity, while image texture descriptors are compared using the Jeffrey divergence operator.

References

  1. Aparajeya P, Leymarie FF (2014) Point-based medialness for animal and plant identification. In: Vrochidis S, et al. (eds) Proceedings of the 1st International Workshop on Environnmental Multimedia Retrieval, vol. 122. CEUR-WS.org, Glasgow, UK, pp 14–21

  2. Arnheim R (1974) Art and Visual Perception: A Psychology of the Creative Eye, new version expanded and revised edition of the 1954 original edn, University of California Press

  3. Bai X, Liu W, Tu Z (2009) Integrating contour and skeleton for shape classification. In: IEEE 12th International Conference on Computer Vision (ICCV) Workshops, pp. 360–367

  4. Bay H, Ess A, Tuytelaars T, Gool LV (2008) Speeded-up robust features (SURF). Comp Vision Image Underst 110(3):346–359

    Article  Google Scholar 

  5. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 24(4):509–522

    Article  Google Scholar 

  6. Berretti S, Bimbo AD, Pala P (2000) Retrieval by shape similarity with perceptual distance and effective indexing. IEEE Transactions on Multimedia 2 (4):225–239

    Article  Google Scholar 

  7. Biederman I (2001) Recognizing depth-rotated objects: A review of recent research and theory. Spatial Vision 13(2–3):241–253

    Google Scholar 

  8. Blum H (1973) Biological shape and visual science. J Theor Biology 38(2):205–287

    Article  Google Scholar 

  9. Bookstein FL (1991) Morphometric Tools for Landmark Data: Geometry and Biology. Cambridge University Press

  10. Bregler C, Loeb L, Chuang E, Deshpande H (2002) Turning to the masters: Motion capturing cartoons. ACM Trans Graph 21(3):399–407

    Article  Google Scholar 

  11. Caputo B et al (2013) ImageCLEF 2013: The vision, the data and the open challenges. In: Information Access Evaluation. Multilinguality, Multimodality, and Visualization, pp. 250–268. Springer

  12. Chen L, Feris R, Turk M (2008) Efficient partial shape matching using Smith-Waterman algorithm. In: Computer Vision and Pattern Recognition Workshops, pp. 1–6

  13. Cope J, Corney D, Clark J, Remagnino P, Wilkin P (2012) Plant species identification using digital morphometrics: A review. Expert Syst Appl 39(8):7562–7573

    Article  Google Scholar 

  14. Dougherty ER, Lotufo RA (2003) Hands-On Morphological Image Processing. Tutorial Texts in Optical Engineering, Vol. TT59. SPIE Publications

  15. Everingham M, Gool LV, Williams CKI, Winn J., Zisserman A (2010) The PASCAL visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  16. Gopalan R, Turaga P, Chellappa R (2010) Articulation-invariant representation of non-planar shapes. In: Computer Vision — ECCV 2010, Lecture Notes in Computer Science (LNCS), vol. 6313, pp. 286–299. Springer

    Chapter  Google Scholar 

  17. Guay M, Cani MP, Ronfard R (2013) The Line of Action: An intuitive interface, ACM Transactions on Graphics 32(6), Article no 205

    Article  Google Scholar 

  18. Heimann T, Meinzer HP (2009) Statistical shape models for 3D medical image segmentation: A review. Med Image Anal 13(4):543–563

    Article  Google Scholar 

  19. Hu RX, Ja W, Zhao Y, Gui J (2012) Perceptually motivated morphological strategies for shape retrieval. Pattern Recognit 45:3222–3230

    Article  Google Scholar 

  20. Kayaert G, Wagemans J, Vogels R (2011) Encoding of complexity, shape, and curvature by macaque infero-temporal neurons, Frontiers in Systems Neuroscience, vol 5

  21. Kelly MF, Levine MD (1995) Annular symmetry operators: A method for locating and describing objects. In: International Conference on Computer Vision (ICCV), pp. 1016–1021

  22. Keustermans J, Vandermeulen D, Mollemans W, Schutyser F, Suetens P (2014) Construction of statistical shape models using a probabilistic point-based shape representation. In: Symposium on Statistical Shape Models and Applications, Article 21. Delémont, Switzerland. http://shapesymposium.org/

  23. Kimia BB (2003) On the role of medial geometry in human vision, Journal of Physiology – Paris 97(2) pp. 155–90

    Article  Google Scholar 

  24. Kovács I (2010) Hot spots and dynamic coordination in Gestalt perception. In: Dynamic Coordination in the Brain: From Neurons to Mind, pp. 215–228. MIT Press

    Chapter  Google Scholar 

  25. Kovács I, Fehér Á, Julesz B (1998) Medial-point description of shape. Vis Res 38(15):2323–2333

    Article  Google Scholar 

  26. Larese MG, Namías R, Craviotto RM, Arango MR, Gallo C, Granitto PM (2014) Automatic classification of legumes using leaf vein image features. Pattern Recog 47:158–168

    Article  Google Scholar 

  27. Latecki LJ, Lakamper R (2000) Shape similarity measure based on correspondence of visual parts. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 22(10):1185–1190

    Article  Google Scholar 

  28. Layton OW, Mingolla E, Yazdanbakhsh A (2014) Neural dynamics of feedforward and feedback processing in figure-ground segregation. Frontiers in Psychology 5(Article 972), 20 pages, Perception Science Series

  29. Leymarie F, Levine MD (1992) Simulating the grassfire transform using an active contour model. IEEE Transactions on Pattern Analysis and Machine Intelligence 14 (1):56–75

    Article  Google Scholar 

  30. Leymarie FF, Aparajeya P, MacGillivray C (2014) Point-based medialness for movement computing. In: Proceedings of the 1st ACM International Workshop on Movement and Computing (MOCO), pp. 31–36

  31. Leyton M (1992) Symmetry, Causality, Mind. MIT Press

  32. Ling H, Jacobs DW (2007) Shape classification using the inner-distance. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(2):286–299

    Article  Google Scholar 

  33. Liu Z, An J, Meng F (2011) A robust point matching algorithm for image registration. In: Fourth International Conference on Machine Vision (ICMV), vol. SPIE 8350

  34. Loomis A (1951) Successful Drawing, Viking Books

  35. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  36. Mingqiang Y, Kidiyo K, Joseph R (2008) A survey of shape feature extraction techniques. In: Yin PY (ed) Pattern Recognition Techniques, Technology and Applications, chap. 3, pp. 43–90. InTech

    Google Scholar 

  37. Mouine S, Yahiaoui I, Verroust-Blondet A (2012) Advanced shape context for plant species identification using leaf image retrieval. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval (ICMR), Article 49. 8 pages

  38. Nanni L, Lumini A, Brahnam S (2014) Ensemble of shape descriptors for shape retrieval and classification. International Journal of Advanced Intelligence Paradigms 6 (2):136–156

    Article  Google Scholar 

  39. Park J, Hwang E, Nam Y (2008) Utilizing venation features for efficient leaf image retrieval. J Syst Softw 81(1):71–82

    Article  Google Scholar 

  40. Pizer SM, Siddiqi K, Székely G, Damon JN, Zucker SW (2003) Multiscale medial loci and their properties. Int J Comput Vis 55(2/3):155–179

    Article  Google Scholar 

  41. Pizer SM, et al. (2003) Deformable M-reps for 3D medical image segmentation. Int J Comput Vis 55(2/3):85–106

    Article  Google Scholar 

  42. Premachandran V, Kakarala R (2013) Perceptually motivated shape context which uses shape interiors. Pattern Recognit 46:2092–2102

    Article  Google Scholar 

  43. Richards W, Hoffman DD (1985) Codon constraints on closed 2D shapes. Computer Vision. Graphics, and Image Processing (CVGIP) 31(3):265–281

    Article  Google Scholar 

  44. Rijsbergen CJV (1979) Information Retrieval, 2nd edn, Butterworth-Heinemann

  45. Roman-Rangel E, Gayol CP, Odobez JM (2011) Searching the past: An improved shape descriptor to retrieve Maya hieroglyphs. In: ACM Multimedia. Scottsdale, Arizona, USA

  46. Sebastian TB, Klein PN, Kimia BB (2004) Recognition of shapes by editing their shock graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(5):550–571

    Article  Google Scholar 

  47. Serra J (ed) (1988) Image Analysis and Mathematical Morphology, vol. 2. Academic Press

  48. Shen W, Wang X, Yao C, Bai X (2014) Shape recognition by combining contour and skeleton into a mid-level representation. In: Li S., Liu C., Wang Y. (eds) Pattern Recognition, vol 483. Springer, pp 391–400

    Google Scholar 

  49. Simmons S, Winer MSA (1977) Drawing: The Creative Process, Simon and Schuster (Prentice-Hall)

  50. Srestasathiern P, Yilmaz A (2011) Planar shape representation and matching under projective transformation. Comp Vision Image Underst (CVIU) 115(11):1525–1535

    Article  Google Scholar 

  51. Tang J, Shao L, Jones S (2014) Point pattern matching based on line graph spectral context and descriptor embedding. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 17–22

  52. van Tonder GJ, Ejima Y (2003) Flexible computation of shape symmetries within the maximal disk paradigm. IEEE Transactions on Systems, Man, and Cybernetics (SMC), Part B: Cybernetics, 33(3), pp. 535–540

  53. Vincent L. (1993) Morphological grayscale reconstruction in image analysis. IEEE Trans Image Process 2(2):176–201

    Article  Google Scholar 

  54. Wamelen PBV, Li Z, Iyengar SS (2004) A fast expected time algorithm for the 2-D point pattern matching problem. Pattern Recognit 37(8):1699–711

    Article  Google Scholar 

  55. Wang X, Feng B, Bai X, Liu W, Latecki LJ (2014) Bag of contour fragments for robust shape classification. Pattern Recognit 47(6):2116–2125

    Article  Google Scholar 

  56. Xie J, Heng PA, Shah M (2008) Shape matching and modeling using skeletal context. Pattern Recognit 41(5):1756–1767

    Article  Google Scholar 

Download references

Acknowledgments

This work was partially funded by the European Union (FP7 – ICT; Grant Agreement #258749; CEEDs project). Thanks to Prof. Stefan Rueger and Prof. Ilona Kovacs for useful discussions.

Author information

Authors and Affiliations

  1. Deparment of Computing, Goldsmiths, University of London, London, SE14 6NW, UK

    Prashant Aparajeya & Frederic Fol Leymarie

Authors
  1. Prashant Aparajeya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frederic Fol Leymarie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Prashant Aparajeya.

Appendix A: F-Measure

Appendix A: F-Measure

In Information Retrieval, the balanced F-score is defined as the harmonic mean of

Precision and Recall [44]:

$$ F-measure=2\times\frac{Precision\times Recall}{Precision+Recall} $$
(15)

Our derived formula of F−measure (11) is based on this equation:

$$F=\frac{2\times(|M_{I}|+|M_{E}|)}{(|Q_{I}|+|Q_{E}|)+(|T_{I}|+|T_{E}|)}\,. $$

The classical definitions of Precision and Recall are given as:

$$ Precision=\frac{t_{p}}{t_{p}+f_{p}} $$
(16)
$$ Recall=\frac{t_{p}}{t_{p}+f_{n}} $$
(17)

where, t p = true positive, f p = false positive, and f n = false negative. Consider the case of internal medial (dominant) and external (concave) points as the input of this evaluation metric, then:

t p = #(feature points matched correctly) = |M I |+|M E |,

f p = #(feature points in the query image that are not matched), and

f n = #(feature points in the target image that are not matched).

Hence:

t p +f p = #(feature points in the query image) = |Q I |+|Q E |, and

t p +f n = #(feature points in the target image) = |T I |+|T E | .

Putting these values in (16) and (17) and further using (15) results in our definition of F-measure (11).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aparajeya, P., Leymarie, F.F. Point-based medialness for 2D shape description and identification. Multimed Tools Appl 75, 1667–1699 (2016). https://doi.org/10.1007/s11042-015-2605-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2605-6

Keywords

Search

Navigation