Skip to main content
SpringerLink
Log in

Reducing the Overconfidence of Base Classifiers when Combining Their Decisions

  • Conference paper
  • First Online:
Multiple Classifier Systems (MCS 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2709))

Included in the following conference series:

Abstract

When the sample size is small, the optimistically biased outputs produced by expert classifiers create serious problems for the combiner rule designer. To overcome these problems, we derive analytical expressions for bias reduction for situations when the standard Gaussian density-based quadratic classifiers serve as experts and the decisions of the base experts are aggregated by the behavior-space-knowledge (BKS) method. These reduction terms diminish the experts’ overconfidence and improve the multiple classification system’s generalization ability. The bias-reduction approach is compared with the standard BKS, majority voting and stacked generalization fusion rules on two real-life datasets for which the different base expert aggregates comprise the multiple classification system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. J. Kittler, F. Roli (eds). Multiple Classifier Systems. Lecture Notes in Computer Science, Springer Vol. 1857 (2000), vol. 2096 (2001), Vol. 2364 (2002).

    Google Scholar 

  2. J. Kittler. Combining classifiers: a theoretical framework. Pattern Analysis and Applications, vol. 1, pp.18–27, 1998

    Article  Google Scholar 

  3. J. Kittler. A framework for classifier fusion: is still needed? Advances in Pattern Recognition, Lecture Notes in Computer Science, Springer, vol. 1876, pp. 45–56, 2000.

    Chapter  Google Scholar 

  4. J. Gosh. Multi-classifier systems: back to the future. Multiple Classifier Systems. Lecture Notes in Computer Science, Springer, vol. 2364, pp. 1–15, 2002.

    Chapter  Google Scholar 

  5. T.K. Ho. Data complexity analysis for classifier combination. Multiple Classifier Systems. Lecture Notes in Computer Science, Springer, vol. 2096, pp. 53–67, 2001.

    Google Scholar 

  6. S. Raudys. Multiple classifier systems in the context of feature extraction and selection. Multiple Classification Systems, Lecture Notes in Computer Science, Springer, vol. 2364, pp. 27–41, 2002.

    Chapter  Google Scholar 

  7. J. Kittler, M. Hatef, R.P.W. Duin, J. Matas. On combining classifiers, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 20, pp. 226–239, 1998.

    Article  Google Scholar 

  8. L.I. Kuncheva, J.C. Bezdek, R.P.W. Duin. Decision templates for multiple classifier fusion: and experimental comparison. Pattern Recognition, vol. 34, pp. 299–314, 2001.

    Article  MATH  Google Scholar 

  9. L. Breiman. Bagging predictors. Machine Learning Journal, vol. 24, pp. 123–140, 1996.

    MATH  MathSciNet  Google Scholar 

  10. L. Breiman. Arcing classifiers. Annals of Statistics, vol. 26, pp. 801–849, 1998.

    Article  MATH  MathSciNet  Google Scholar 

  11. Y. Freund, R.E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and Systems Sci., vol. 55, pp. 119–139, 1997.

    Article  MATH  MathSciNet  Google Scholar 

  12. M. Jordan and R. Jakobs. Hierarchical mixture of experts and the EM algorithm. Neural Computation, vol. 6, pp. 181–214, 1994.

    Article  Google Scholar 

  13. D. Wolpert. Stacked generalization. Neural Networks, vol. 5, pp. 241–260, 1992.

    Article  Google Scholar 

  14. F. Roli, G. Fumera. Analysis of linear and order statistics combiners for fusion of imbalanced classifiers. Multiple Classification Systems, Lecture Notes in ComputerScience, Springer, vol. 2364, pp. 252–261, 2002.

    Article  Google Scholar 

  15. F. Roli, S. Raudys, G.L. Marcialis. An experimental comparison of fixed and trained rules for crisp classifiers outputs, Multiple Classification Systems, Lecture Notes in Computer Science, Springer, vol. 2364, pp. 232–241, 2002.

    Chapter  Google Scholar 

  16. C. Güler, B. Sankur, Y. Kahya, M. Skurichina, S. Raudys. Classification of respiratory sound patterns by means of cooperative neural networks. In: G. Ramponi, G. L. Sicuranza, S. Carrato, S. Marsi (eds), Proc. of 8th European Signal Processing Conference (isbn 88-86179-83-9). Edizioni Lint, Trieste, 1996.

    Google Scholar 

  17. A. Janeliunas, S. Raudys. Reduction of boasting bias’ of linear expert. Multiple Classification Systems, Springer, Lecture Notes in Comp. Science, vol. 2364, pp. 242–251,2002.

    Article  Google Scholar 

  18. S. Raudys. Experts’ boasting in trainable fusion rules. IEEE Transactions on Pattern Analysis and Machine Intelligence (in press, 2003).

    Google Scholar 

  19. S. Raudys. Statistical and Neural Classifiers: An integrated approach to design. Springer, London, p. 312, 2001.

    MATH  Google Scholar 

  20. P.A. Lachenbruch, M. Goldstein. Discriminant analysis. Biometrics, vol. 5, pp. 9–85, 1979.

    MathSciNet  Google Scholar 

  21. Y.S. Huang, C.Y. Suen. A method of combining multiple experts for the recognition of unconstrained handwritten numerals. IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 17, pp. 90–93, 1998.

    Article  Google Scholar 

  22. L.I. Kuncheva, C.J. Whitaker. Feature subsets for classifier combination: An enumerative experiment. Multiple Classifier Systems. Lecture Notes in Computer Science, Springer, vol. 2096, pp. 228–237, 2001.

    Google Scholar 

  23. K. Fukunaga. Introduction to Statistical Pattern Recognition. 2nd ed. Academic Press, New York, 1990.

    MATH  Google Scholar 

  24. R.O. Duda, P.E. Hart, D.G. Stork. Pattern Classification. 2nd ed. Wiley, NY, 2000.

    Google Scholar 

  25. S. Raudys, (1972). On the amount of a priori information in designing the classification algorithm. Proc. Acad. of Sciences of the USSR, 168–174, (in Russian).

    Google Scholar 

  26. S. Raudys and D. Young. A review of the former Soviet Union literature on small training sample problems in statistical discriminant analysis. Journal of Multivariate Analysis. (in press 2003).

    Google Scholar 

  27. A.E. Nikulin, B. Dolenko, T. Bezabeh, R.L. Somorjai. Near-optimal region selection for feature space reduction: novel preprocessing methods for classifying MR spectra. NMR Biomed. 11, 209–216, 1998.

    Article  Google Scholar 

  28. C.L. Lean, R.L Somorjai, I.C.P. Smith, P. Russell, C.E. Mountford. Accurate diagnosis and prognosis of human cancers by proton MRS and a three stage classification strategy. Annual Reports on NMR Spectroscopy, vol. 48, 71–111, 2002.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Vilnius Gediminas Technical University, Sauletekio 11, Vilnius, LT-2100, Lithuania

    Šarunas Raudys

  2. Institute for Biodiagnostics, National Research Council Canada, 435 Ellice Avenue, Winnipeg, MB, Canada, R3B 1Y6

    Ray Somorjai & Richard Baumgartner

Authors
  1. Šarunas Raudys
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ray Somorjai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richard Baumgartner
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford, Surrey, GU2 7XH, UK

    Terry Windeatt

  2. Dept. of Electrical and Electronic Engineering, University of Cagliari, Piazza D’Ami, 09123, Cagliari, Italy

    Fabio Roli

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Raudys, Š., Somorjai, R., Baumgartner, R. (2003). Reducing the Overconfidence of Base Classifiers when Combining Their Decisions. In: Windeatt, T., Roli, F. (eds) Multiple Classifier Systems. MCS 2003. Lecture Notes in Computer Science, vol 2709. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44938-8_7

Download citation

  • DOI: https://doi.org/10.1007/3-540-44938-8_7

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40369-2

  • Online ISBN: 978-3-540-44938-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation