Pattern Recognition and String Matching pp 673-702 | Cite as

# Enhanced Neighbourhood Specifications for Pattern Classification

## Abstract

Automatic classification is an active research area in artificial intelligence, pattern recognition, and machine learning. Its goal is to make a computer automatically label attribute vectors as examples of two or more different categories or classes. One of the most widely studied approaches corresponds to the well known Nearest Neighbour (NN) decision rule, proposed by Fix and Hodges [19], formally analysed by Cover and Hart [9], and then investigated both theoretically and experimentally by many researchers. On the other hand, a lot of successful applications have been reported in a multitude of real-world domains [10].

### Keywords

Editing Univer## Preview

Unable to display preview. Download preview PDF.

### References

- [1]A. Akkus and H.A. Güvenir,
*k*Nearest neighbor classification on feature projections, in L. Saitta (ed.)*Proc. of the 13th International Conference on Machine Learning*, (Morgan Kaufmann, Bari, Italy, 1996 ) pp. 12–19.Google Scholar - [2]R. Barandela and E. Gasca, Decontamination of training samples for supervised pattern recognition methods. in F.J. Ferri et al. (eds.)
*Advances in Pattern Recognition*, ( Springer Verlag, Alicante, Spain, 2000 ) pp. 621–630.CrossRefGoogle Scholar - [3]S.O. Belkasim, M. Shridhar, and M. Ahmadi, Pattern classification using an efficient KNNR,
*Pattern Recognition*Vol. 25, (1992) pp. 1269–1274.CrossRefGoogle Scholar - [4]A.L. Blum and P. Langley, Selection of relevant features and examples in machine learning,
*Artificial Intelligence*Vol. 97, (1997) pp. 245–271.CrossRefMATHMathSciNetGoogle Scholar - [5]C.E. Brodley and M.A. Friedl, Identifying mislabeled training data,
*Journal of Artificial Intelligence Research*Vol. 11, (1999) pp. 131–167.MATHGoogle Scholar - [6]C.L. Chang, Finding prototypes for nearest neighbor classifiers.
*IEEE Trans. on Computer*s Vol. 23 (1974) pp. 1179–1184.CrossRefMATHGoogle Scholar - [7]B.B. Chaudhuri, A new definition of neighborhood of a point in multidimensional space,
*Pattern Recognition Letters*Vol. 17, (1996) pp. 11–17.CrossRefGoogle Scholar - [8]S. Cost and S. Salzberg, A weighted nearest neighbor algorithm for learning with symbolic features,
*Machine Learning*Vol. 10, (1993) pp. 57–78.Google Scholar - [9]T.M. Cover and P.E. Hart, Nearest neighbor pattern classification,
*IEEE Trans. on Information Theory*Vol. 13, (1967) pp. 21–27.CrossRefMATHGoogle Scholar - [10]B.V. Dasarathy,
*Nearest Neighbor Norms: NN Pattern Classification techniques*, (IEEE Computer Society Press, Los Alamos, CA, 1991 ).Google Scholar - [11]B.V. Dasarathy, Minimal consistent subset (MCS) identification for optimal nearest neighbor decision systems design,
*IEEE Trans. on Systems, Man, and Cybernetics*Vol. 24, (1994), pp. 511–517.CrossRefGoogle Scholar - [12]B.V. Dasarathy, J.S. Sanchez, and S. Townsend, Nearest neighbor editing and condensing tools-synergy exploitation,
*Pattern Analysis and Applications*Vol. 3, (2000) pp. 19–30.CrossRefGoogle Scholar - [13]M. de Berg, M. van Kreveld, M. Overmars, and O. Schwarzkopf,
*Computational Geometry: Algorithms and Applications*, ( Springer-Verlag, Berlin, 1997 ).MATHGoogle Scholar - [14]P.A. Devijver and J. Kittler,
*Pattern Recognition: A Statistical Approach*, (Prentice Hall, Englewood Cliffs, NJ, 1982 ).MATHGoogle Scholar - [15]A. Djouadi and E. Bouktache, A fast algorithm for the nearest-neighbor classifier,
*IEEE Trans. on Pattern Analysis and Machine Intelligence*Vol. 19, (1997) pp. 277–282.CrossRefGoogle Scholar - [16]A. Faragó, T. Linder and G. Lugosi, Fast nearest-neighbor search in dissimilarity spaces,
*IEEE Trans. on Pattern Analysis and Machine Intelligence*Vol. 15, (1993) pp. 957–962.CrossRefGoogle Scholar - [17]F.J. Ferri, J.S. Sanchez, and F. Pla, Editing prototypes in the finite sample size case using alternative neighbourhoods, in A. Amin et al. (eds.)
*Advances in Pattern Recognition*, ( Springer-Verlag, Sydney, Australia, 1998 ) pp 620–629.CrossRefGoogle Scholar - [18]F.J. Ferri, J.V. Albert, and E. Vidal, Considerations about sample-size sensitivity of a family of edited nearest-neighbor rules,
*IEEE Trans. on Systems, Man, and Cybernetics-Part B: Cybernetics*Vol. 29, (1999) pp. 667–672.CrossRefGoogle Scholar - [19]E. Fix and J.L. Hodges, Discriminatory analysis, non–parametric discrimination, consistency properties,
*Project 21–49–004, Report No. 4, Contract AF41(128)–3*, (USAF School of Aviation Medicine, Randolph Field, TX, 1951 ).Google Scholar - [20]K. Fukunaga and P.M. Narendra, A branch and bound algorithm for computing k-nearest neighbors,
*IEEE Trans. on Computers*Vol. 24 (1975) pp. 750–753.CrossRefMATHMathSciNetGoogle Scholar - [21]K. Fukunaga and P.M. Narendra, A branch and bound algorithm for feature subset selection,
*IEEE Trans. on Computers*Vol. 26 (1977) pp. 917–922.CrossRefGoogle Scholar - [22]G.W. Gates, The reduced nearest neighbor rule,
*IEEE Trans. on Information Theory*Vol. 18, (1972) pp. 431–433.CrossRefGoogle Scholar - [22]G.W. Gates, The reduced nearest neighbor rule,
*IEEE Trans. on Information Theory*Vol. 18, (1972) pp. 431–433.CrossRefGoogle Scholar - [24]D.J. Hand, J.N. Kok, and M.R. Berthold,
*Advances in Intelligent Data Analysis*, ( Springer Verlag, Berlin, 1999 ).CrossRefGoogle Scholar - [25]P.E. Hart, The condensed nearest neighbor rule,
*IEEE Trans. on Information Theory*Vol. 14, (1968) pp. 515–516.CrossRefGoogle Scholar - [26]K. Hattori and M. Takahashi, A new edited k-nearest neighbor rule in the pattern classification problem,
*Pattern Recognition*Vol. 33, (2000) pp. 521–528.CrossRefGoogle Scholar - [27]M.E. Hellman, The nearest neighbor classification rule with a reject option,
*IEEE Trans. on Systems, Science, and Cybernetics*Vol. 6, (1970) pp. 179–185.CrossRefMathSciNetGoogle Scholar - [28]A.K. Jain
*Fundamentals of Digital Image Processing*, (Prentice Hall, Englewood Cliffs, NJ, 1989 ).MATHGoogle Scholar - [29]A. Jain and D. Zongker, Feature selection: evaluation, application and small sample performance,
*IEEE Trans. on Pattern Analysis and Machine Intelligence*Vol. 19, (1997) pp. 153–158.CrossRefGoogle Scholar - [30]J.W. Jaromczyk and G.T. Toussaint, Relative neighborhood graphs and their relatives,
*Proc. IEEE*Vol. 80, (1992) pp. 1502–1517.CrossRefGoogle Scholar - [31]G.H. John, R. Kohavi, and K. Pfleger, Irrelevant features and the subset selection problem, in W.W. Cohen and H. Hirsh (eds.)
*Proc. of the 11th International Conference on Machine Learning*, ( Morgan Kaufmann, New Brunswick, NJ, 1994 ) pp. 121–129.Google Scholar - [32]A. Józwik and G. Vernazza, Recognition of leucocytes by a parallel
*k*-NN classifier, in*Lecture Notes of ICB Seminar*, ( Warsaw, Poland, 1988 ) pp. 138–153.Google Scholar - [33]B.S. Kim and S.B. Park, A fast nearest neighbor finding algorithm based on the ordered partition,
*IEEE Trans. on Pattern Analysis and Machine Intelligence Vol.*8 (1986) pp. 761–766.CrossRefMATHGoogle Scholar - [34]R. Kohavi and G.H. John, Wrappers for feature subset selection,
*Artificial Intelligence*Vol. 97, (1997) pp. 273–324.CrossRefMATHGoogle Scholar - [35]J. Koplowitz, and T.A. Brown, On the relation of performance to editing in nearest neighbor rules.,
*Pattern Recognition*Vol. 13, (1981) pp. 251–255.CrossRefGoogle Scholar - [36]L.I. Kuncheva, Editing for the
*k*-nearest neighbors rule by a genetic algorithm,*Pattern Recognition Letters*Vol. 16, (1995) pp. 809–814.CrossRefGoogle Scholar - [37]L.I. Kuncheva and L.C. Jair, Nearest neighbor classifier: simultaneous editing and feature selection,
*Pattern Recognition Letters*Vol. 20, (1999) pp. 1149–1156.CrossRefGoogle Scholar - [38]P. Langley and W. Iba, Average-case analysis of a nearest neighbor algorithm, in R. Bajcsy (ed.)
*Proc. of the 13th International Joint Conference on Artificial Intelligence*, ( Morgan Kaufmann, Chambéry, France, 1993 ) pp. 889–894.Google Scholar - [39]C.X. Ling and H. Wang, Computing optimal attribute weight settings for nearest neighbor algorithms,
*Artificial Intelligence Review*Vol.11, (1997) pp. 255–272.CrossRefGoogle Scholar - [40]C.J. Merz and P.M. Murphy,
*UCI Repository of Machine Learning Databases*. (Univ. of California, Irvine, http://www.ics.uci.edu/mlearn, 1998).Google Scholar - [41]L. Micó, J. Oncina and R.C. Carrasco, A fast branch and bound nearest neighbour classifier in metric spaces,
*Pattern Recognition Letters*Vol. 17 (1996) pp. 731–739.CrossRefGoogle Scholar - [42]F. Moreno-Seco, J.M. Inesta, L. Micó, and J. Oncina, Fast
*k*-neighbour classification of human vertebrae levels, in J.S. Sanchez and F. Pla (ed.),*Pattern Recognition and Image Analysis II*, ( Publicacions de la Universitat Jaume I, Castellón, Spain, 2001 ) pp. 343–348.Google Scholar - [43]S.A. Nene and S.K. Nayar, A simple algorithm for nearest neighbor search in high dimensions,
*IEEE Trans. on Pattern Analysis and Machine Intelligence*Vol. 19 (1997) pp. 989–1003.CrossRefGoogle Scholar - [44]J.F. O’Callaghan, An alternative definition for neighborhood of a point,
*IEEE Trans. on Computer*s Vol. 24, (1975) pp. 1121–1125.CrossRefMATHGoogle Scholar - [45]F.P. Preparata and M.I. Shamos,
*Computational Geometry. An introduction*, (Springer, New York, 1985 ).Google Scholar - [46]P. Pudil, J. Novovicova and J. Kittler, Floating search methods in feature selection,
*Pattern Recognition Letters*Vol. 15 (1994) pp. 1119–1125.CrossRefGoogle Scholar - [47]V. Ramasubramanian and K.K. Paliwal, Fast nearest-neighbor search algorithms based on approximation-elimination search,
*Pattern Recognition*Vol. 33 (2000) pp. 1497–1510.CrossRefGoogle Scholar - [48]G.L. Ritter, H.B. Woodritz, S.R. Lowry and T.L. Isenhour, An algorithm for selective nearest neighbor rule.
*IEEE Trans. on Information Theory*Vol. 21 (1975) pp. 665–669.CrossRefMATHGoogle Scholar - [49]J.S. Sanchez, F. Pla, and F.J. Ferri, Prototype selection for the nearest neighbour rule through proximity graphs,
*Pattern Recognition Letters*Vol. 18, (1997) pp. 507–513.CrossRefGoogle Scholar - [50]J.S. Sanchez, F. Pla, and F.J. Ferri, On the use of neighbourhood-based non-parametric classifiers,
*Pattern Recognition Letters*Vol. 18, (1997) pp. 1179–1186.CrossRefGoogle Scholar - [51]J.S. Sanchez, F. Pla, and F.J. Ferri, Improving the k-NCN classification rule through heuristic modifications,
*Pattern Recognition Letters*Vol. 19, (1998) pp. 1165–1170.CrossRefMATHGoogle Scholar - [52]J.S. Sánchez, R. Barandela, R. Alejo, and A.I. Marqués, Performance evaluation of prototype selection algorithms for nearest neighbor classification, in D.L. Borges and S.-T. Wu (eds.)
*Proc. of the 14th. Brazilian Symposium on Computer Graphics and Image Processing*, ( IEEE Computer Society Press, Florianópolis, Brazil, 2001 ) pp. 44–50.CrossRefGoogle Scholar - [53]J.S. Sánchez, R. Barandela, A.I. Marqués, R. Alejo, and J. Badenas, Analysis of new techniques to obtain quality trainning sets,
*Pattern Recognition Letters*, (2001) (*in press*).Google Scholar - [54]W. Siedlecki and J. Sklansky, On automatic feature selection,
*International Journal of Pattern Recognition and Artificial Intelligence*Vol. 2, (1988) pp. 197–220.CrossRefGoogle Scholar - [55]I. Tomek, An experiment with the edited nearest neighbor rule,
*IEEE Trans. on Systems, Man, and Cybernetics*Vol. 6, (1976) pp. 448–452.CrossRefMATHMathSciNetGoogle Scholar - [56]G.T. Toussaint, B.K. Bhattacharya, and R.S. Poulsen, The application of Voronoi diagrams to nonparametric decision rules, in L. Billard (ed.)
*Proc. of the 16th Symposium on the Interface: Computer Science and Statistics*, ( Atlanta, GA, 1984 ) pp. 97–108.Google Scholar - [57]J.I. Toriwaki and S. Yokoi, Voronoi and related neighbors on digitized two dimensional space with applications to texture analysis, in G.T. Toussaint (ed.)
*Computational Morphology*, (North-Holland, 1988 ) pp. 207–228.Google Scholar - [58]M. Tüceryan and T. Chorzempa, Relative sensitivity of a family of closest-point graphs in computer vision applications,
*Pattern Recognition*Vol. 24, (1991) pp. 361–373.CrossRefGoogle Scholar - [59]E. Vidal, An algorithm for finding nearest neighbours in (approximately) constant average time,
*Pattern Recognition Letters*Vol. 4, (1986) pp. 145–147.CrossRefGoogle Scholar - [60]J.M. Vilar, Reducing the overhead of the AESA metric-space nearest neighbour searching algorithm,
*Information Processing Letters*Vol. 56 (1995) pp. 265–271.CrossRefMATHMathSciNetGoogle Scholar - [61]D.L. Wilson, Asymptotic properties of nearest neighbor rules using edited data,
*IEEE Trans. on Systems, Man, and Cybernetics*Vol. 2, (1972) pp. 408–421.CrossRefMATHGoogle Scholar - [62]D.R. Wilson and T.R. Martinez, Reduction techniques for instance-based learning algorithms.
*Machine Learning*Vol. 38 (2000) pp. 257–286.CrossRefMATHGoogle Scholar - [63]T. Young and K.-S. Fu,
*Handbook of Pattern Recognition and Image Processing*, (Academic Press, Orlando, FL, 1986 ).MATHGoogle Scholar - [64]J. Zhang, Y. Yim, and J. Yang, Intelligent selection of instances for prediction functions in lazy learning algorithms,
*Artificial Intelligence Review*Vol. 11, (1997) pp. 175–191.CrossRefGoogle Scholar