Combination of Classifiers

  • M. Narasimha Murty
  • V. Susheela Devi
Part of the Undergraduate Topics in Computer Science book series (UTICS, volume 0)


A combination or an ensemble of classifiers is a set of classifiers whose individual decisions are combined to classify new examples. A combination of classifiers is often much more accurate than the individual classifiers that make them up. One reason for this could be that the training data may not provide sufficient information for choosing a single best classifier and a combination is the best compromise. Another reason could be that the learning algorithms used may not be able to solve the difficult search problem posed. Since solving the search problem may be difficult, suitable heuristics may be used in the search. As a consequence of this, even though with the training examples and prior knowledge, a unique best hypothesis exists, we may not be able to find it. A combination of classifiers is a way of compensating for imperfect classifiers. The learning algorithms we use may give us good approximations to the true value but may not be the right hypothesis. By taking a weighted combination of these approximations, we may be able to represent the true hypothesis. In fact, the combination could be equivalent to very complex decision trees.


Markov Chain Monte Carlo Majority Vote Test Pattern Markov Chain Monte Carlo Method Classifier Ensemble 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bryll, Robert, Ricardo Gutierrez-Osuna and Francis Quek. Attribute bagging: Improving accuracy of classifier ensembles by using random feature subsets. Pattern Recognition 36(6): 1291–1302. 2003.MATHCrossRefGoogle Scholar
  2. 2.
    Chen, Lei and Mohamed S. Karnel. A generalized adaptive ensemble generation and aggregation approach for multiple classifier systems. Pattern Recognition 42(5): 629–644. 2009.MATHCrossRefGoogle Scholar
  3. 3.
    Dietterich, T.G. Machine learning research: Four current directions. AI Magazine 18(4): 97–136. 1997.Google Scholar
  4. 4.
    Drucker, H., C. Cortes, L. D. Jackel, Y. Lecun, V. Vapnik. Boosting and other ensemble methods. Neural Computation 6(6): 1289–1301. 1994.MATHCrossRefGoogle Scholar
  5. 5.
    Freund, Yoav and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55(1): 119–139. 1997.MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Freund, Yoav and Robert E. Schapire. A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence 14(5): 771–780. 1999.Google Scholar
  7. 7.
    Ho, Tin Kam, Jonathan J. Hull and Sargur N. Srihari. Decision combination in multiple classifier systems. IEEE Trans on PAMI 16(1). 1994.Google Scholar
  8. 8.
    Hothorn, Torsten and Berthold Lausen. Double-bagging: Combining classifiers by bootstrap aggregation. Pattern Recognition 36(6): 1303–1309. 2003.MATHCrossRefGoogle Scholar
  9. 9.
    Huang, Y. S. and C.Y. Suen. A method of combining multiple experts for the recognition of unconstrained handwritten numerals. IEEE Trans. on Pattern Analysis and Machine Intelligence 17: 90–94. 1995.CrossRefGoogle Scholar
  10. 10.
    Santos, Eulanda M. Dos, Robert Sabourin and Patrick Maupin. A dynamic overproduce-and-choose strategy for the selection of classifier ensembles. Pattern Recognition 41(10): 2993–3009. 2008.MATHCrossRefGoogle Scholar
  11. 11.
    Saranli, Afsar and Mbeccel Demirekler. A statistical unified framework for rank-based multiple classifier decision combination. Pattern Recognition 34(4): 865–884. 2001.MATHCrossRefGoogle Scholar
  12. 12.
    Wang, Xiao and Han Wang. Classification by evolutionary ensembles. Pattern Recognition 39(4): 595–607. 2006.MATHCrossRefGoogle Scholar
  13. 13.
    Windeatt, Terry. Vote counting measures for ensemble classifiers. Pattern Recognition 36(12): 2743–2756. 2003.MATHCrossRefGoogle Scholar
  14. 14.
    Xu, Lei, Adam Krzyzak and Ching Y. Suen. Methods of combining multiple classifiers and their application to handwriting recognition. IEEE Trans on SMC 22(3). 1992.Google Scholar
  15. 15.
    Zouari, Hla, Laurent Heutte and Yves Lecourtier. Controlling the diversity in classifier ensembles through a measure of agreement. Pattern Recognition 38(11): 2195–2199. 2005.CrossRefGoogle Scholar

Copyright information

© Universities Press (India) Pvt. Ltd. 2011

Authors and Affiliations

  1. 1.Dept. of Computer Science and AutomationIndian Institute of ScienceBangaloreIndia

Personalised recommendations