Skip to main content
SpringerLink
Log in

On the Number of Modes of a Gaussian Mixture

  • Conference paper
  • First Online:
Scale Space Methods in Computer Vision (Scale-Space 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2695))

Included in the following conference series:

Abstract

We consider a problem intimately related to the creation of maxima under Gaussian blurring: the number of modes of a Gaussian mixture in D dimensions. To our knowledge, a general answer to this question is not known. We conjecture that if the components of the mixture have the same covariance matrix (or the same covariance matrix up to a scaling factor), then the number of modes cannot exceed the number of components. We demonstrate that the number of modes can exceed the number of components when the components are allowed to have arbitrary and different covariance matrices.

We will review related results from scale-space theory, statistics and machine learning, including a proof of the conjecture in 1D. We present a convergent, EM-like algorithm for mode finding and compare results of searching for all modes starting from the centers of the mixture components with a brute-force search. We also discuss applications to data reconstruction and clustering.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Carreira-Perpiñán, M.Á., Williams, C.K.I.: On the number of modes of a Gaussian mixture. Technical Report EDI-INF-RR-0159, School of Informatics, University of Edinburgh, UK (2003). Available online at http://www.informatics.ed.ac.uk/publications/report/0159.html.

    Google Scholar 

  2. Carreira-Perpiñán, M.Á.: Continuous Latent Variable Models for Dimensionality Reduction and Sequential Data Reconstruction. PhD thesis, Dept. of Computer Science, University of Sheffield, UK (2001)

    Google Scholar 

  3. Carreira-Perpiñán, M.Á.: Mode-finding for mixtures of Gaussian distributions. Technical Report CS-99-03, Dept. of Computer Science, University of Sheffield, UK (1999), revised August 4, 2000. Available online at http://www.dcs.shef.ac.uk/~miguel/papers/cs-99-03.html.

    Google Scholar 

  4. Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Computation 14 (2002) 1771–1800

    Article  MATH  Google Scholar 

  5. Lindeberg, T.: Scale-Space Theory in Computer Vision. Kluwer Academic Publishers Group, Dordrecht, The Netherlands (1994)

    Google Scholar 

  6. Koenderink, J.J.: The structure of images. Biol. Cybern. 50 (1984) 363–370

    Article  MATH  MathSciNet  Google Scholar 

  7. Babaud, J., Witkin, A.P., Baudin, M., Duda, R.O.: Uniqueness of the Gaussian kernel for scale-space filtering. IEEE Trans. on Pattern Anal. and Machine Intel. 8 (1986) 26–33

    MATH  Google Scholar 

  8. Yuille, A.L., Poggio, T.A.: Scaling theorems for zero crossings. IEEE Trans. on Pattern Anal. and Machine Intel. 8 (1986) 15–25

    Article  MATH  Google Scholar 

  9. Roberts, S.J.: Parametric and non-parametric unsupervised cluster analysis. Pattern Recognition 30 (1997) 261–272

    Article  Google Scholar 

  10. Lifshitz, L.M., Pizer, S.M.: A multiresolution hierarchical approach to image segmentation based on intensity extrema. IEEE Trans. on Pattern Anal. and Machine Intel. 12 (1990) 529–540

    Article  Google Scholar 

  11. Kuijper, A., Florack, L.M.J.: The application of catastrophe theory to image analysis. Technical Report UU-CS-2001-23, Dept. of Computer Science, Utrecht University (2001). Available online at ftp://ftp.cs.uu.nl/pub/RUU/CS/techreps/CS-2001/2001-23.pdf.

    Google Scholar 

  12. Kuijper, A., Florack, L.M.J.: The relevance of non-generic events in scale space models. In Heyden, A., Sparr, G., Nielsen, M., Johansen, P., eds.: Proc. 7th European Conf. Computer Vision (ECCV’02), Copenhagen, Denmark (2002)

    Google Scholar 

  13. Damon, J.: Local Morse theory for solutions to the heat equation and Gaussian blurring. J. Diff. Equations 115 (1995) 368–401

    Article  MATH  MathSciNet  Google Scholar 

  14. Silverman, B.W.: Using kernel density estimates to investigate multimodality. Journal of the Royal Statistical Society, B 43 (1981) 97–99

    Google Scholar 

  15. Carreira-Perpiñán, M.Á.: Mode-finding for mixtures of Gaussian distributions. IEEE Trans. on Pattern Anal. and Machine Intel. 22 (2000) 1318–1323

    Article  Google Scholar 

  16. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, B 39 (1977) 1–38

    MATH  MathSciNet  Google Scholar 

  17. McLachlan, G.J., Krishnan, T.: The EM Algorithm and Extensions. Wiley Series in Probability and Mathematical Statistics. John Wiley & Sons (1997)

    Google Scholar 

  18. Redner, R.A., Walker, H.F.: Mixture densities, maximum likelihood and the EM algorithm. SIAM Review 26 (1984) 195–239

    Article  MATH  MathSciNet  Google Scholar 

  19. Rose, K.: Deterministic annealing for clustering, compression, classification, regression, and related optimization problems. Proc. IEEE 86 (1998) 2210–2239

    Article  Google Scholar 

  20. Schölkopf, B., Mika, S., Burges, C.J.C., Knirsch, P., Müller, K.R., Rätsch, G., Smola, A.: Input space vs. feature space in kernel-based methods. IEEE Trans. Neural Networks 10 (1999) 1000–1017

    Article  Google Scholar 

  21. Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Computation 3 (1991) 79–87

    Article  Google Scholar 

  22. Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, New York, Oxford (1995)

    Google Scholar 

  23. Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Number 26 in Monographs on Statistics and Applied Probability. Chapman & Hall, London, New York (1986)

    MATH  Google Scholar 

  24. Carreira-Perpiñán, M.Á.: Reconstruction of sequential data with probabilistic models and continuity constraints. In Solla, S.A., Leen, T.K., Müller, K.R., eds.: Advances in Neural Information Processing Systems. Volume 12, MIT Press, Cambridge, MA (2000) 414–420

    Google Scholar 

  25. Bishop, C.M., Svensén, M., Williams, C.K.I.: GTM: The generative topographic mapping. Neural Computation 10 (1998) 215–234

    Article  Google Scholar 

  26. Fukunaga, K., Hostetler, L.D.: The estimation of the gradient of a density function, with application in pattern recognition. IEEE Trans. Inf. Theory IT-21 (1975) 32–40

    Article  MathSciNet  Google Scholar 

  27. Cheng, Y.: Mean shift, mode seeking, and clustering. IEEE Trans. on Pattern Anal. and Machine Intel. 17 (1995) 790–799

    Article  Google Scholar 

  28. Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. IEEE Trans. on Pattern Anal. and Machine Intel. 24 (2002) 603–619

    Article  Google Scholar 

  29. Wong, Y.: Clustering data by melting. Neural Computation 5 (1993) 89–104

    Article  Google Scholar 

  30. Chakravarthy, S.V., Ghosh, J.: Scale-based clustering using the radial basis function network. IEEE Trans. Neural Networks 7 (1996) 1250–1261

    Article  Google Scholar 

  31. Leung, Y., Zhang, J.S., Xu, Z.B.: Clustering by scale-space filtering. IEEE Trans. on Pattern Anal. and Machine Intel. 22 (2000) 1396–1410

    Article  Google Scholar 

  32. Minnotte, M.C., Scott, D.W.: The mode tree: A tool for visualization of nonparametric density features. Journal of Computational and Graphical Statistics 2 (1993) 51–68

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Computer Science, University of Toronto, Toronto

    Miguel Á. Carreira-Perpiñán

  2. School of Informatics, University of Edinburgh, Edinburgh

    Christopher K. I. Williams

Authors
  1. Miguel Á. Carreira-Perpiñán
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher K. I. Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Imaging Sciences, 5th Floor Thomas Guy House, Guy’s Campus, London, SE1 9RT, UK

    Lewis D. Griffin

  2. IT University of Copenhagen, Glentevej 67-69, Copenhagen NV, Denmark

    Martin Lillholm

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Carreira-Perpiñán, M.Á., Williams, C.K.I. (2003). On the Number of Modes of a Gaussian Mixture. In: Griffin, L.D., Lillholm, M. (eds) Scale Space Methods in Computer Vision. Scale-Space 2003. Lecture Notes in Computer Science, vol 2695. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44935-3_44

Download citation

  • DOI: https://doi.org/10.1007/3-540-44935-3_44

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40368-5

  • Online ISBN: 978-3-540-44935-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation