Nearest Neighbour Based Classifiers

Part of the Undergraduate Topics in Computer Science book series (UTICS, volume 0)


One of the simplest decision procedures that can be used for classification is the nearest neighbour (NN) rule. It classifies a sample based on the category of its nearest neighbour. When large samples are involved, it can be shown that this rule has a probability of error which is less than twice the optimum error—hence there is less than twice the probability of error compared to any other decision rule. The nearest neighbour based classifiers use some or all the patterns available in the training set to classify a test pattern. These classifiers essentially involve finding the similarity between the test pattern and every pattern in the training set.


Leaf Node Class Label Test Pattern Neighbour Search Near Neighbour 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Broder, A. J. Strategies for efficient incremental nearest neighbour search. Pattern Recognition 23(1/2): 171–178. 1990.CrossRefGoogle Scholar
  2. 2.
    Chang, C. L. Finding prototypes for nearest neighbour classifiers. IEEE Trans. on Computers C-23(11): 1179–1184. 1974.CrossRefGoogle Scholar
  3. 3.
    Cover, T. M. and P. E. Hart. Nearest neighbor pattern classification IEEE Trans. on Information Theory IT-13: 21–27. 1967.CrossRefGoogle Scholar
  4. 4.
    Dasarathy, Belur V. Minimal consistent set (MCS) identification for optimal nearest neighbour decision system design. IEEE Trans. on Systems, Man and Cybernetics 24(3). 1994.Google Scholar
  5. 5.
    Dejiver, P. A. and J. Kittler. On the edited nearest neighbour rule. Proceedings of the 5th International Conference on Pattern Recognition. pp. 72–80. 1980.Google Scholar
  6. 6.
    Dudani, S. A. The distance-weighted k nearest neighbor rule. IEEE Trans. on SMC SMC-6(4): 325–327. 1976.Google Scholar
  7. 7.
    Friedman, J. H., F. Baskett and L. J. Shustek. An algorithm for finding nearest neighbours. IEEE Trans on Computers C-24(10): 1000–1006. 1975.CrossRefGoogle Scholar
  8. 8.
    Fukunaga, K. and P. M. Narendra. A branch and bound algorithm for computing k nearest neighbours. IEEE Trans. on Computers. pp. 750–753. 1975.Google Scholar
  9. 9.
    Gates, G. W. The reduced nearest neighbour rule. IEEE Trans. on Information Theory IT-18(3): 431–433. 1972.CrossRefGoogle Scholar
  10. 10.
    Gowda, K.C. and G. Krishna. Edit and error correction using the concept of mutual nearest neighbourhood. International Conference on Cybernetics and Society. pp. 222–226. 1979.Google Scholar
  11. 11.
    Hart, P. E. The condensed nearest neighbor rule. IEEE Trans. on Information Theory IT-14(3): 515–516. 1968.CrossRefGoogle Scholar
  12. 12.
    Jozwik, A. A learning scheme for a fuzzy kNN rule. Pattern Recognition Letters 1(5/6): 287–289. 1983.CrossRefGoogle Scholar
  13. 13.
    Kim, B. S. and S. B. Park. A fast nearest neighbour finding algorithm based on the ordered partition. IEEE Trans on PAMI PAMI-8(6): 761–766. 1986.CrossRefGoogle Scholar
  14. 14.
    Kuncheva, L. Editing for the k nearest neighbours rule by a genetic algorithm. Pattern Recognition Letters 16(8): 809–814. 1995.CrossRefGoogle Scholar
  15. 15.
    Kuncheva, L. and L. C. Jain. Nearest neighbor classifier: Simultaneous editing and feature selection. Pattern Recognition Letters 20: 1149–1156. 1999.CrossRefGoogle Scholar
  16. 16.
    Lai, Jim Z. C., Yi-Ching Liaw and Julie Liu. Fast k nearest neighbour search based on projection and triangular inequality. Pattern Recognition 40: 351–359. 2007.MATHCrossRefGoogle Scholar
  17. 17.
    McNames, James. A fast nearest neighbour algorithm based on a principal axis search tree. IEEE Trans. on Pattern Analysis and Machine Intelligence 23(9): 964–976. 2001.CrossRefGoogle Scholar
  18. 18.
    Miclet, L. and M.Dabouz. Approximative fast nearest neighbour recognition. Pattern Recognition Letters 1: 277–285. 1983.CrossRefGoogle Scholar
  19. 19.
    Papadimitriou, C. H. and J. L. Bentley. A worst-case analysis of nearest neighbour searching by projection. Lecture Notes in Computer Science 85: 470–482. 1980.MathSciNetGoogle Scholar
  20. 20.
    Patrick, E. A., and F. P. Fischer. A generalized k nearest neighbor rule. Information and Control 16: 128–152. 1970.MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Sanchez, J. S., F. Pla and F. J. Ferri. Prototype selection for the nearest neighbour rule through proximity graphs. Pattern Recognition Letters 18(6): 507–513. 1995.CrossRefGoogle Scholar
  22. 22.
    Devi, V. Susheela and M. Narasimha Murty. An incremental prototype set building technique. Pattern Recognition 35: 505–513. 2002.MATHCrossRefGoogle Scholar
  23. 23.
    Swonger, C.W. Sample set condensation for a condensed nearest neighbor decision rule for pattern recognition. Frontiers of Pattern Recognition. 511– 519. 1972.Google Scholar
  24. 24.
    Tomek, I. A generalization of the kNNrule. IEEE Trans. on SMC SMC-6(2): 121–126. 1976.MathSciNetGoogle Scholar
  25. 25.
    Tomek, I. An experiment with the edited nearest neighbour rule. IEEE Trans. on SMC SMC-6(6): 448–452. 1976.MathSciNetGoogle Scholar
  26. 26.
    Lam, Wai, Chi-Kin Keung and Charles X. Ling. Learning good prototypes for classification using filtering and abstraction of instances. Pattern Recognition 35: 1491–1506. 2002.MATHCrossRefGoogle Scholar
  27. 27.
    Lam, Wai, Chi-Kin Keung and Danyu Liu. Discovering useful concept prototypes for classification based on filtering and abstraction. IEEE Trans PAMI 24(8): 1075–1090. 2002.CrossRefGoogle Scholar
  28. 28.
    Wilson, D. L. Asymptotic properties of nearest neighbour rules using edited data. IEEE Trans. SMC SMC-2(3): 408–421. 1972.Google Scholar
  29. 29.
    Yunck, Thomas P. A technique to identify nearest neighbours. IEEE Trans. SMC SMC-6(10): 678–683. 1976.MathSciNetGoogle Scholar
  30. 30.
    Zhang, Bin and Sargur N. Srihari. Fast k nearest neighbour classification using cluster-based trees. IEEE Trans. on Pattern Analysis and Machine Intelligence 26(4): 525–528. 2004.CrossRefGoogle Scholar

Copyright information

© Universities Press (India) Pvt. Ltd. 2011

Authors and Affiliations

  1. 1.Dept. of Computer Science and AutomationIndian Institute of ScienceBangaloreIndia

Personalised recommendations