Enhanced Neighbourhood Specifications for Pattern Classification

  • J. S. Sánchez
  • A. I. Marqués
Part of the Combinatorial Optimization book series (COOP, volume 13)

Abstract

Automatic classification is an active research area in artificial intelligence, pattern recognition, and machine learning. Its goal is to make a computer automatically label attribute vectors as examples of two or more different categories or classes. One of the most widely studied approaches corresponds to the well known Nearest Neighbour (NN) decision rule, proposed by Fix and Hodges [19], formally analysed by Cover and Hart [9], and then investigated both theoretically and experimentally by many researchers. On the other hand, a lot of successful applications have been reported in a multitude of real-world domains [10].

Keywords

Editing Univer 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    A. Akkus and H.A. Güvenir, k Nearest neighbor classification on feature projections, in L. Saitta (ed.) Proc. of the 13th International Conference on Machine Learning, (Morgan Kaufmann, Bari, Italy, 1996 ) pp. 12–19.Google Scholar
  2. [2]
    R. Barandela and E. Gasca, Decontamination of training samples for supervised pattern recognition methods. in F.J. Ferri et al. (eds.) Advances in Pattern Recognition, ( Springer Verlag, Alicante, Spain, 2000 ) pp. 621–630.CrossRefGoogle Scholar
  3. [3]
    S.O. Belkasim, M. Shridhar, and M. Ahmadi, Pattern classification using an efficient KNNR, Pattern Recognition Vol. 25, (1992) pp. 1269–1274.CrossRefGoogle Scholar
  4. [4]
    A.L. Blum and P. Langley, Selection of relevant features and examples in machine learning, Artificial Intelligence Vol. 97, (1997) pp. 245–271.CrossRefMATHMathSciNetGoogle Scholar
  5. [5]
    C.E. Brodley and M.A. Friedl, Identifying mislabeled training data, Journal of Artificial Intelligence Research Vol. 11, (1999) pp. 131–167.MATHGoogle Scholar
  6. [6]
    C.L. Chang, Finding prototypes for nearest neighbor classifiers. IEEE Trans. on Computer s Vol. 23 (1974) pp. 1179–1184.CrossRefMATHGoogle Scholar
  7. [7]
    B.B. Chaudhuri, A new definition of neighborhood of a point in multidimensional space, Pattern Recognition Letters Vol. 17, (1996) pp. 11–17.CrossRefGoogle Scholar
  8. [8]
    S. Cost and S. Salzberg, A weighted nearest neighbor algorithm for learning with symbolic features, Machine Learning Vol. 10, (1993) pp. 57–78.Google Scholar
  9. [9]
    T.M. Cover and P.E. Hart, Nearest neighbor pattern classification, IEEE Trans. on Information Theory Vol. 13, (1967) pp. 21–27.CrossRefMATHGoogle Scholar
  10. [10]
    B.V. Dasarathy, Nearest Neighbor Norms: NN Pattern Classification techniques, (IEEE Computer Society Press, Los Alamos, CA, 1991 ).Google Scholar
  11. [11]
    B.V. Dasarathy, Minimal consistent subset (MCS) identification for optimal nearest neighbor decision systems design, IEEE Trans. on Systems, Man, and Cybernetics Vol. 24, (1994), pp. 511–517.CrossRefGoogle Scholar
  12. [12]
    B.V. Dasarathy, J.S. Sanchez, and S. Townsend, Nearest neighbor editing and condensing tools-synergy exploitation, Pattern Analysis and Applications Vol. 3, (2000) pp. 19–30.CrossRefGoogle Scholar
  13. [13]
    M. de Berg, M. van Kreveld, M. Overmars, and O. Schwarzkopf, Computational Geometry: Algorithms and Applications, ( Springer-Verlag, Berlin, 1997 ).MATHGoogle Scholar
  14. [14]
    P.A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach, (Prentice Hall, Englewood Cliffs, NJ, 1982 ).MATHGoogle Scholar
  15. [15]
    A. Djouadi and E. Bouktache, A fast algorithm for the nearest-neighbor classifier, IEEE Trans. on Pattern Analysis and Machine Intelligence Vol. 19, (1997) pp. 277–282.CrossRefGoogle Scholar
  16. [16]
    A. Faragó, T. Linder and G. Lugosi, Fast nearest-neighbor search in dissimilarity spaces, IEEE Trans. on Pattern Analysis and Machine Intelligence Vol. 15, (1993) pp. 957–962.CrossRefGoogle Scholar
  17. [17]
    F.J. Ferri, J.S. Sanchez, and F. Pla, Editing prototypes in the finite sample size case using alternative neighbourhoods, in A. Amin et al. (eds.) Advances in Pattern Recognition, ( Springer-Verlag, Sydney, Australia, 1998 ) pp 620–629.CrossRefGoogle Scholar
  18. [18]
    F.J. Ferri, J.V. Albert, and E. Vidal, Considerations about sample-size sensitivity of a family of edited nearest-neighbor rules, IEEE Trans. on Systems, Man, and Cybernetics-Part B: Cybernetics Vol. 29, (1999) pp. 667–672.CrossRefGoogle Scholar
  19. [19]
    E. Fix and J.L. Hodges, Discriminatory analysis, non–parametric discrimination, consistency properties, Project 21–49–004, Report No. 4, Contract AF41(128)–3, (USAF School of Aviation Medicine, Randolph Field, TX, 1951 ).Google Scholar
  20. [20]
    K. Fukunaga and P.M. Narendra, A branch and bound algorithm for computing k-nearest neighbors, IEEE Trans. on Computers Vol. 24 (1975) pp. 750–753.CrossRefMATHMathSciNetGoogle Scholar
  21. [21]
    K. Fukunaga and P.M. Narendra, A branch and bound algorithm for feature subset selection, IEEE Trans. on Computers Vol. 26 (1977) pp. 917–922.CrossRefGoogle Scholar
  22. [22]
    G.W. Gates, The reduced nearest neighbor rule, IEEE Trans. on Information Theory Vol. 18, (1972) pp. 431–433.CrossRefGoogle Scholar
  23. [22]
    G.W. Gates, The reduced nearest neighbor rule, IEEE Trans. on Information Theory Vol. 18, (1972) pp. 431–433.CrossRefGoogle Scholar
  24. [24]
    D.J. Hand, J.N. Kok, and M.R. Berthold, Advances in Intelligent Data Analysis, ( Springer Verlag, Berlin, 1999 ).CrossRefGoogle Scholar
  25. [25]
    P.E. Hart, The condensed nearest neighbor rule, IEEE Trans. on Information Theory Vol. 14, (1968) pp. 515–516.CrossRefGoogle Scholar
  26. [26]
    K. Hattori and M. Takahashi, A new edited k-nearest neighbor rule in the pattern classification problem, Pattern Recognition Vol. 33, (2000) pp. 521–528.CrossRefGoogle Scholar
  27. [27]
    M.E. Hellman, The nearest neighbor classification rule with a reject option, IEEE Trans. on Systems, Science, and Cybernetics Vol. 6, (1970) pp. 179–185.CrossRefMathSciNetGoogle Scholar
  28. [28]
    A.K. Jain Fundamentals of Digital Image Processing, (Prentice Hall, Englewood Cliffs, NJ, 1989 ).MATHGoogle Scholar
  29. [29]
    A. Jain and D. Zongker, Feature selection: evaluation, application and small sample performance, IEEE Trans. on Pattern Analysis and Machine Intelligence Vol. 19, (1997) pp. 153–158.CrossRefGoogle Scholar
  30. [30]
    J.W. Jaromczyk and G.T. Toussaint, Relative neighborhood graphs and their relatives, Proc. IEEE Vol. 80, (1992) pp. 1502–1517.CrossRefGoogle Scholar
  31. [31]
    G.H. John, R. Kohavi, and K. Pfleger, Irrelevant features and the subset selection problem, in W.W. Cohen and H. Hirsh (eds.) Proc. of the 11th International Conference on Machine Learning, ( Morgan Kaufmann, New Brunswick, NJ, 1994 ) pp. 121–129.Google Scholar
  32. [32]
    A. Józwik and G. Vernazza, Recognition of leucocytes by a parallel k -NN classifier, in Lecture Notes of ICB Seminar, ( Warsaw, Poland, 1988 ) pp. 138–153.Google Scholar
  33. [33]
    B.S. Kim and S.B. Park, A fast nearest neighbor finding algorithm based on the ordered partition, IEEE Trans. on Pattern Analysis and Machine Intelligence Vol. 8 (1986) pp. 761–766.CrossRefMATHGoogle Scholar
  34. [34]
    R. Kohavi and G.H. John, Wrappers for feature subset selection, Artificial Intelligence Vol. 97, (1997) pp. 273–324.CrossRefMATHGoogle Scholar
  35. [35]
    J. Koplowitz, and T.A. Brown, On the relation of performance to editing in nearest neighbor rules., Pattern Recognition Vol. 13, (1981) pp. 251–255.CrossRefGoogle Scholar
  36. [36]
    L.I. Kuncheva, Editing for the k -nearest neighbors rule by a genetic algorithm, Pattern Recognition Letters Vol. 16, (1995) pp. 809–814.CrossRefGoogle Scholar
  37. [37]
    L.I. Kuncheva and L.C. Jair, Nearest neighbor classifier: simultaneous editing and feature selection, Pattern Recognition Letters Vol. 20, (1999) pp. 1149–1156.CrossRefGoogle Scholar
  38. [38]
    P. Langley and W. Iba, Average-case analysis of a nearest neighbor algorithm, in R. Bajcsy (ed.) Proc. of the 13th International Joint Conference on Artificial Intelligence, ( Morgan Kaufmann, Chambéry, France, 1993 ) pp. 889–894.Google Scholar
  39. [39]
    C.X. Ling and H. Wang, Computing optimal attribute weight settings for nearest neighbor algorithms, Artificial Intelligence Review Vol.11, (1997) pp. 255–272.CrossRefGoogle Scholar
  40. [40]
    C.J. Merz and P.M. Murphy, UCI Repository of Machine Learning Databases. (Univ. of California, Irvine, http://www.ics.uci.edu/mlearn, 1998).Google Scholar
  41. [41]
    L. Micó, J. Oncina and R.C. Carrasco, A fast branch and bound nearest neighbour classifier in metric spaces, Pattern Recognition Letters Vol. 17 (1996) pp. 731–739.CrossRefGoogle Scholar
  42. [42]
    F. Moreno-Seco, J.M. Inesta, L. Micó, and J. Oncina, Fast k-neighbour classification of human vertebrae levels, in J.S. Sanchez and F. Pla (ed.), Pattern Recognition and Image Analysis II, ( Publicacions de la Universitat Jaume I, Castellón, Spain, 2001 ) pp. 343–348.Google Scholar
  43. [43]
    S.A. Nene and S.K. Nayar, A simple algorithm for nearest neighbor search in high dimensions, IEEE Trans. on Pattern Analysis and Machine Intelligence Vol. 19 (1997) pp. 989–1003.CrossRefGoogle Scholar
  44. [44]
    J.F. O’Callaghan, An alternative definition for neighborhood of a point, IEEE Trans. on Computer s Vol. 24, (1975) pp. 1121–1125.CrossRefMATHGoogle Scholar
  45. [45]
    F.P. Preparata and M.I. Shamos, Computational Geometry. An introduction, (Springer, New York, 1985 ).Google Scholar
  46. [46]
    P. Pudil, J. Novovicova and J. Kittler, Floating search methods in feature selection, Pattern Recognition Letters Vol. 15 (1994) pp. 1119–1125.CrossRefGoogle Scholar
  47. [47]
    V. Ramasubramanian and K.K. Paliwal, Fast nearest-neighbor search algorithms based on approximation-elimination search, Pattern Recognition Vol. 33 (2000) pp. 1497–1510.CrossRefGoogle Scholar
  48. [48]
    G.L. Ritter, H.B. Woodritz, S.R. Lowry and T.L. Isenhour, An algorithm for selective nearest neighbor rule. IEEE Trans. on Information Theory Vol. 21 (1975) pp. 665–669.CrossRefMATHGoogle Scholar
  49. [49]
    J.S. Sanchez, F. Pla, and F.J. Ferri, Prototype selection for the nearest neighbour rule through proximity graphs, Pattern Recognition Letters Vol. 18, (1997) pp. 507–513.CrossRefGoogle Scholar
  50. [50]
    J.S. Sanchez, F. Pla, and F.J. Ferri, On the use of neighbourhood-based non-parametric classifiers, Pattern Recognition Letters Vol. 18, (1997) pp. 1179–1186.CrossRefGoogle Scholar
  51. [51]
    J.S. Sanchez, F. Pla, and F.J. Ferri, Improving the k-NCN classification rule through heuristic modifications, Pattern Recognition Letters Vol. 19, (1998) pp. 1165–1170.CrossRefMATHGoogle Scholar
  52. [52]
    J.S. Sánchez, R. Barandela, R. Alejo, and A.I. Marqués, Performance evaluation of prototype selection algorithms for nearest neighbor classification, in D.L. Borges and S.-T. Wu (eds.) Proc. of the 14th. Brazilian Symposium on Computer Graphics and Image Processing, ( IEEE Computer Society Press, Florianópolis, Brazil, 2001 ) pp. 44–50.CrossRefGoogle Scholar
  53. [53]
    J.S. Sánchez, R. Barandela, A.I. Marqués, R. Alejo, and J. Badenas, Analysis of new techniques to obtain quality trainning sets,Pattern Recognition Letters, (2001) (in press).Google Scholar
  54. [54]
    W. Siedlecki and J. Sklansky, On automatic feature selection, International Journal of Pattern Recognition and Artificial Intelligence Vol. 2, (1988) pp. 197–220.CrossRefGoogle Scholar
  55. [55]
    I. Tomek, An experiment with the edited nearest neighbor rule, IEEE Trans. on Systems, Man, and Cybernetics Vol. 6, (1976) pp. 448–452.CrossRefMATHMathSciNetGoogle Scholar
  56. [56]
    G.T. Toussaint, B.K. Bhattacharya, and R.S. Poulsen, The application of Voronoi diagrams to nonparametric decision rules, in L. Billard (ed.) Proc. of the 16th Symposium on the Interface: Computer Science and Statistics, ( Atlanta, GA, 1984 ) pp. 97–108.Google Scholar
  57. [57]
    J.I. Toriwaki and S. Yokoi, Voronoi and related neighbors on digitized two dimensional space with applications to texture analysis, in G.T. Toussaint (ed.) Computational Morphology, (North-Holland, 1988 ) pp. 207–228.Google Scholar
  58. [58]
    M. Tüceryan and T. Chorzempa, Relative sensitivity of a family of closest-point graphs in computer vision applications, Pattern Recognition Vol. 24, (1991) pp. 361–373.CrossRefGoogle Scholar
  59. [59]
    E. Vidal, An algorithm for finding nearest neighbours in (approximately) constant average time, Pattern Recognition Letters Vol. 4, (1986) pp. 145–147.CrossRefGoogle Scholar
  60. [60]
    J.M. Vilar, Reducing the overhead of the AESA metric-space nearest neighbour searching algorithm, Information Processing Letters Vol. 56 (1995) pp. 265–271.CrossRefMATHMathSciNetGoogle Scholar
  61. [61]
    D.L. Wilson, Asymptotic properties of nearest neighbor rules using edited data, IEEE Trans. on Systems, Man, and Cybernetics Vol. 2, (1972) pp. 408–421.CrossRefMATHGoogle Scholar
  62. [62]
    D.R. Wilson and T.R. Martinez, Reduction techniques for instance-based learning algorithms. Machine Learning Vol. 38 (2000) pp. 257–286.CrossRefMATHGoogle Scholar
  63. [63]
    T. Young and K.-S. Fu, Handbook of Pattern Recognition and Image Processing, (Academic Press, Orlando, FL, 1986 ).MATHGoogle Scholar
  64. [64]
    J. Zhang, Y. Yim, and J. Yang, Intelligent selection of instances for prediction functions in lazy learning algorithms, Artificial Intelligence Review Vol. 11, (1997) pp. 175–191.CrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • J. S. Sánchez
    • 1
  • A. I. Marqués
    • 2
  1. 1.Department of Computer Languages and Information SystemsUniversity Jaume ICastellónSpain
  2. 2.Department of Business Administration and MarketingUniversity Jaume ICastellónSpain

Personalised recommendations