Optimum decision rules in pattern recognition

  • Thien M. Ha
Rejection in Pattern Recognition
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1451)


This paper reviews various optimum decision rules for pattern recognition, namely, Bayes rule, Chow's rule (optimum error-reject tradeoff), and the recently proposed class-selective rejection rule. The last one provides the optimum tradeoff between the error rate and the average number of (selected) classes. The usage of each of these rules as well as their relationship are discussed. Some common properties to these rules are pointed out, e.g. the linear time complexity.

Key Words

classification decision rule Bayes rule Chow's rule class-selective rejection rule nearest neighbour rules man-machine interface preselection time complexity 


  1. [1]
    U.S. Baird and C.L. Mallows, “Bounded-Error Preclassification Trees,” in Shape, Structure and Pattern Recognition, D. Dori and A. Bruckstein (Eds.), World Scientific, 1995, pp. 343–349.Google Scholar
  2. [2]
    J.O. Berger, Statistical Decision Theory and Bayesian Analysis, second edition, Springer-Verlag, 1985.Google Scholar
  3. [3]
    R. Chellappa, C.L. Wilson, and S. Sirohey, “Human and Machine Recognition of Faces: A Survey,” Proceedings of the IEEE, Vol. 83, No. 5, pp. 705–740, May 1995.CrossRefGoogle Scholar
  4. [4]
    W. Cho, S.W. Lee, and J.H. Kim, “Modeling and Recognition of Cursive Words with Hidden Markov Models,” Pattern Recognition 28, pp. 1941–1953, 1995.CrossRefGoogle Scholar
  5. [5]
    C.K. Chow, “An Optimum Character Recognition System Using Decision Functions,” Institute of Radio Engineers (IRE,) Transactions on Electronic, Computers, Vol. EC-6, No. 4, pp. 247–254, December 1957.Google Scholar
  6. [6]
    C.K. Chow, “On Optimum Recognition Error and Reject Tradeoff,” IEEE Transactions on Information Theory, Vol. IT-16, No. 1, pp. 41–46, January 1970.CrossRefGoogle Scholar
  7. [7]
    C.K. Chow, “Recognition Error and Reject Trade-off,” Third Annual Symp. on Document Analysis and Information Retrieval, April 11–13, 1994, University of Nevada, Las Vegas, U.S.A., pp. 1–8.Google Scholar
  8. [8]
    T.M. Cover and P.E. Hart, “Nearest Neighbor Pattern Classification,” IEEE Transactions on Information Theory, Vol. 13, pp. 21–27, Jan. 1967.CrossRefGoogle Scholar
  9. [9]
    P.A. Devijver, “Error and Reject Tradeoff for Nearest Neighbor Decision Rules,” in G. Tacconi (Ed.) Aspects of Signal Processing, Part 2, D. Reidel Publishing Company, Dordrecht-Holland, pp. 525–538, 1977.Google Scholar
  10. [10]
    B. Dubuisson and M. Masson, “A Statistical Decision Rule with Incomplete Knowledge about Classes,” Pattern Recognition 26, pp. 155–165, 1993.CrossRefGoogle Scholar
  11. [11]
    R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis, John Wiley & Sons, 1973.Google Scholar
  12. [12]
    G.M. Fitzmaurice and D.J. Hand, “A Comparison of Two Average Conditional Error Rate Estimators,” Pattern Recognition Letters, Vol. 6, pp. 221–224, 1987.CrossRefGoogle Scholar
  13. [13]
    K. Fukunaga and D.L. Kessel, “Application of Optimum Error-Reject Functions,” IEEE Transactions on Information Theory, Vol. IT-18, pp. 814–817, November 1972.CrossRefGoogle Scholar
  14. [14]
    K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition, Academic Press, 1990.Google Scholar
  15. [15]
    Thien M. Ha, D. Niggeler, H. Bunke, and J. Clarinval, “Giro Form Reading Machine,” Optical Engineering, Vol. 34, No. 8, pp. 2277–2288, 1995.Google Scholar
  16. [16]
    Thien M. Ila, “An Optimum Class-Selective Rejection Rule for Pattern Recognition,” Proceedings of the 13 th International Conference on Pattern Recognition, Vol. 11, Aug. 25–30, 1996, Vienna, Austria, pp. 75–80.Google Scholar
  17. [17]
    Thien M. Ila, “On Functional Relation between Class-Selective Rejection Error and Average Number of Classes,” IEEE International Symposia on Intelligence and Systents, Nov. 4–5, 1996, Rockville, Maryland, U.S.A., pp. 282–287.Google Scholar
  18. [18]
    Thien M. Ha, “The Optimum Class-Selective Rejection Rule,” IEEE Traits. on Pattern Analysis and Machine Intelligence. Vol. 19, No. 6, pp. 608–615, June 1997.CrossRefGoogle Scholar
  19. [19]
    D.J. Hand, “Recent Advances in Error Rate Estimation,” Pattern Recognition Letters, Vol. 4, pp. 335–346, 1986.CrossRefGoogle Scholar
  20. [20]
    D.J. Hand, “An Optimal Error Rate Estimator Based on Average Conditional Error Rate: Asymptotic Results,” Pattern Recognition Letters, Vol. 4, pp. 347–350, 1986.CrossRefGoogle Scholar
  21. [21]
    M.E. Hellman, “The Nearest Neighbor Classification Rule with a Reject Option,” IEEE Transactions on Systems, Science, and Cybernetics, Vol. SSC-6, No. 3, pp. 179–185, July 1970.Google Scholar
  22. [22]
    C.G.Y. Lau (Editor), Neural Networks: Theoretical Foundations and Analysis, IEEE Press, 1992.Google Scholar
  23. [23]
    G. Lugosi and M. Pawlak, “On the Posterior-Probability Estimate of the Error Rate of Nonparametric Classification Rules,” IEEE Transactions on Information Theory, Vol. IT-40, No.2, pp. 475–481, March 1994.CrossRefGoogle Scholar
  24. [24]
    M. Pawlak, “On the Asymptotic Properties of Smoothed Estimators of the Classification Error Rate,” Pattern Recognition, Vol. 21, No. 5, pp. 515–524, 1988.CrossRefGoogle Scholar
  25. [25]
    L. Rabiner and B.H. Juang, Fundamentals of Speech Recognition, Prentice-Hall, 1993.Google Scholar
  26. [26]
    B.D. Ripley, Pattern Recognition and Neural Networks, Cambridge University Press, 1996.Google Scholar
  27. [27]
    S.M. Ross, A First Course in Probability, third edition, Macmillan Publishing Company, 1988.Google Scholar
  28. [28]
    J. Schürmann, Pattern Classification: A Unified View of Statistical and Neural Approaches, John Wiley & Sons, 1996.Google Scholar
  29. [29]
    G.T. Toussaint, “Bibliography on Estimation of Misclassifications,” IEEE Transactions on Information Theory, Vol. IT-20, pp. 472–479, July 1974.CrossRefGoogle Scholar
  30. [30]
    G.E. Tutz, “Smoothed Additive Estimators for Non-Error Rates in Multiple Discriminant Analysis,” Pattern Recognition, Vol. 18, No. 2, pp. 151–159, 1985.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Thien M. Ha
    • 1
  1. 1.Institut für Informatik und Angewandte MathematikUniversity of BerneBerneSwitzerland

Personalised recommendations