Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Multiple Classifier Systems

MCS 2003: Multiple Classifier Systems pp 1–14Cite as

Data Dependence in Combining Classifiers

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2709))

Abstract

It has been accepted that multiple classifier systems provide a platform for not only performance improvement, but more efficient and robust pattern classification systems. A variety of combining methods have been proposed in the literature and some work has focused on comparing and categorizing these approaches. In this paper we present a new categorization of these combining schemes based on their dependence on the data patterns being classified. Combining methods can be totally independent from the data, or they can be implicitly or explicitly dependent on the data. It is argued that data dependent, and especially explicitly data dependent, approaches represent the highest potential for improved performance. On the basis of this categorization, an architecture for explicit data dependent combining methods is discussed. Experimental results to illustrate the comparative performance of some combining methods according to this categorization is included.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. J. Kittler, and F. Roli (Eds.), “Multiple classifier systems”, First International Workshop, MCS2000, Cagliari, Italy, June 21–23, 2000, Proceedings, Vol. 1857 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 2000.

    Google Scholar 

  2. J. Kittler, and F. Roli (Eds.), “Multiple classifier systems”, Second International Workshop, MCS2001, Cambridge, UK, July 2–4, 2001 Proceedings, Vol. 2096 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 2001.

    Google Scholar 

  3. F. Roli, and J. Kittler (Eds.), “Multiple classifier systems”, Third International Workshop, MCS2002, Cagliari, Italy June 24–26, 2002, Proceedings, Vol. 2364 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 2002.

    Google Scholar 

  4. J. Kittler, M. Hatef, D. Robert, J. Matas, “On combining classifiers”. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20:3, pp. 226–239, 1998.

    Article  Google Scholar 

  5. A. Sharakey, “Types of multinet systems”, In: F. Roli, and J. Kittler (Eds.), “Multiple classifier systems”, Third International Workshop, MCS2002, Cagliari, Italy June 24–26, 2002, Proceedings, Vol. 2364 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, pp. 108–117, 2002.

    Chapter  Google Scholar 

  6. A. Sharkey, “Multinet Systems”, In: A. Sharkey (Ed.), “Combining Artificial Neural Nets”, Springer-Verlag, Berlin, pp. 1–30, 1999.

    Google Scholar 

  7. G. Auda, and M. Kamel, “Modular neural network classifiers: A comparative study”, Journal of Intelligent and Robotic Systems, Vol. 21, pp. 117–129, 1998.

    Article  Google Scholar 

  8. N. Wanas, and M. Kamel, “Combining neural network ensembles”, International Joint Conference on Neural Networks (IJCNN’01), Washington, D.C., Jul 15 — 19, pp. 2952–2957, 2001.

    Google Scholar 

  9. D. Wolpert, “Stacked generalization”, Neural Networks, Vol. 5, pp. 241–259, 1992.

    Article  Google Scholar 

  10. T. Ho, J. Hull, S. Srihari, “Decision combination in multiple classifier systems”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 16:1, pp. 66–75, 1994.

    Article  Google Scholar 

  11. L. Kuncheva, “A theoritical study on six classifier fusion strategies”, IEEE Transactions on Systems, Man and Cybernetics, Vol. 24:2, pp. 281–286, 2002.

    Google Scholar 

  12. A. Verikas, A. Lipnickas, K. Malmqvist, M. Bacauskiene, A. Gelzinis, “Soft combination of neural classifiers: A comparative study”, Pattern Recognition Letters, Vol. 20, pp. 429–444, 1999.

    Article  Google Scholar 

  13. R. Duin, and D. Tax, “Experiments with Classifier Combining Rules”, In: J. Kittler, and F. Roli (Eds.), “Multiple classifier systems”, First International Workshop, MCS2000, Cagliari, Italy, June 21–23, 2000, Proceedings, Vol. 1857 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, pp. 16–29, 2000.

    Chapter  Google Scholar 

  14. L. Alexandre, A. Campihlo, and M. Kamel, “On combining classifiers using sum and product rules”, Pattern Recognition Letters, Vol. 22, pp. 1283–1289, 2001.

    Article  MATH  Google Scholar 

  15. D. Tax, M. Breukelen, R. Duin, and J. Kittler, “Combining multiple classifiers by averaging or by mutliplying?”, Pattern Recognition, Vol 33, pp. 1475–1485, 2000.

    Article  Google Scholar 

  16. R. Duin, “The combining classifier: To train or not to train?”, Proceedings of the 16th International Conference on Pattern Recognition (ICPR 2002), Quebec City, Canada, Vol 2, pp. 765–770, 2002.

    MathSciNet  Google Scholar 

  17. S. Hashem, “Optimal linear combinations of neural networks”, Neural Networks Vol. 10:4, pp. 599–614, 1997.

    Article  Google Scholar 

  18. N. Ueda, “Optimal Linear Combination of neural networks for improving classification performance”, Pattern Analysis and Machine Intelligence, Vol. 22, No. 2, pp. 207–215, 2000.

    Article  MathSciNet  Google Scholar 

  19. P. Gader, M. Mohamed, and J. Keller, “Fusion of handwritten word classifiers”, Pattern Recognition Letters, Vol. 17, pp. 577–584, 1996.

    Article  Google Scholar 

  20. L. Xu, A. Krzyzk, C. Suen, “Methods of combining multiple classifiers and their application to handwriting recognition”, IEEE Transactions on Systems, Man and Cybernetics, Vol. 22:3, pp. 418–435, 1992.

    Article  Google Scholar 

  21. Y. Freund, and R. Schapire, “Experiments with a new boosting algorithm”, In Proceedings of the 13th International Conference on Machine Learning, Morgan Kaufmann, pp. 149–156, 1996.

    Google Scholar 

  22. Y. Huang, and C. Suen, “A method of combining multiple experts for recognition of unconstrained handwritten numerals”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 17:1, pp. 90–94, 1995.

    Article  Google Scholar 

  23. S. Kumar, J. Ghosh, and M. Crawford, “Hierarchical fusion of multiple classifiers for Hyperspectral Data Analysis”, Pattern Analysis and Applications, Vol. 5, pp. 210–220, 2002.

    Article  MATH  MathSciNet  Google Scholar 

  24. K. Sirlantzis, S. Hoque, and M. C. Fairhurst, “Trainable multiple classifier schemes for handwritten character recognition”, In: F. Roli, J. Kittler (Eds.), “Multiple classifier systems”, Third International Workshop, MCS2002, Cagliari, Italy June 24–26, 2002, Proceedings, Vol. 2364 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, pp. 169–178, 2002.

    Chapter  Google Scholar 

  25. L. Lam, and C. Suen, “Optimal combination of pattern classifiers”, Pattern Recognition Letters, Vol. 16, pp. 945–954, 1995.

    Article  Google Scholar 

  26. L. Kuncheva, J. Bezdek, and R. Duin, “Decsion templates for multiple classifier fusion: an experimental comparision”, Pattern Recognition, Vol. 34, pp. 299–314, 2001.

    Article  MATH  Google Scholar 

  27. L. Kuncheva, “Switching between selection and fusion in combining classifiers: An experiment”, IEEE Transactions on Systems, Man and Cybernetics-Part B, Vol. 32:2, pp. 146–156, 2002.

    Article  Google Scholar 

  28. K. Woods, W. Kegelmeyer, and K. Bowyer, “Combination of multiple classifiers using local accuracy estimates”, Pattern Analysis and Machine Intelligence, Vol. 19, No. 4, pp. 405–410, 1997.

    Article  Google Scholar 

  29. G. Giancinto, and F. Roli, “Dynamic classifier selection based on multiple classifier behaviour”, Pattern Recognition, Vol. 34, pp. 1879–1881, 2001.

    Article  Google Scholar 

  30. M. Jordon, and R. Jacobs, “Hierarchical mixtures of experts and the EM algorithm”, Neural Computing Vol. 6:2, pp. 181–214, 1994.

    Article  Google Scholar 

  31. X. Song, Y. AbuMostafa, J. Sill, H. Kasdan, and M. Pavel, “Robust image recognition by fusion of contextual information”, Information Fusion, Vol. 3, pp. 277–287, 2002.

    Article  Google Scholar 

  32. N. Wanas, L. Hodge, and M. Kamel, “Adaptive Training Algorithm for an Ensemble of Networks”, International Joint Conference on Neural Networks (IJCNN’01), Washington, D.C., Jul 15–19, pp. 2590–2595, 2001.

    Google Scholar 

  33. S. Hashem, “Treating Harmful Collinearity in Neural Network Ensembles”, In: A. Sharkey (Ed.), “Combing Artificial Neural Nets”, Springer-Verlag, Berlin, pp., 1999.

    Google Scholar 

  34. A. Sharkey, N. Sharkey, U. Gerecke, and G. Chandroth, “The test and select approach to ensemble combination”, In: J. Kittler, and F. Roli (Eds.), “Multiple classifier systems”, First International Workshop, MCS2000, Cagliari, Italy, June 21–23, 2000, Proceedings, Vol. 1857 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, pp. 30–44, 2000.

    Chapter  Google Scholar 

  35. Giorgio Giancinto, and Fabio Roli, “An approach to the automatic design of multiple classifier systems”, Pattern Recognition Letters, Vol. 22, pp. 25–33, 2001

    Article  Google Scholar 

  36. C. Blake, and C. Merz, “UCI Repository of machine learning databases [http://www.ics.uci.edu/~mlearn/MLRepository.html]”, University of California, Irvine, Dept. of Information and Computer Sciences, 1998.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Pattern Analysis and Machine Intelligence lab Department of Systems Design Engineering, University of Waterloo, Waterloo, Ontario, Canada, N2L-3G1

    Mohamed S. Kamel & Nayer M. Wanas

Authors
  1. Mohamed S. Kamel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nayer M. Wanas
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford, Surrey, GU2 7XH, UK

    Terry Windeatt

  2. Dept. of Electrical and Electronic Engineering, University of Cagliari, Piazza D’Ami, 09123, Cagliari, Italy

    Fabio Roli

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kamel, M.S., Wanas, N.M. (2003). Data Dependence in Combining Classifiers. In: Windeatt, T., Roli, F. (eds) Multiple Classifier Systems. MCS 2003. Lecture Notes in Computer Science, vol 2709. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44938-8_1

Download citation

  • DOI: https://doi.org/10.1007/3-540-44938-8_1

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40369-2

  • Online ISBN: 978-3-540-44938-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation