Advertisement

Engineering Principles of Pattern Recognition

Chapter

Abstract

In recent years our very complex and technically oriented society has created a situation in which more people and organizations have become concerned with handling information and fewer with handling materials. The need for improved information systems has become more conspicuous, since the world is generating more information in its various forms and information is an essential element in decision making. One of the major problems in the design of modern information systems is automatic pattern recognition. This has been the subject of investigation by many diverse groups, including research workers dealing with electronic computers, automatic controls, information theory, applied physics, statistics, psychology, biology, physiology, medicine, and linguistics. Each group emphasizes certain aspects of the problem. This chapter attempts to discuss some engineering principles for the design of pattern recognition systems.

Keywords

Decision Function Measurement Vector Decision Boundary Pattern Classification Pattern Point 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    K. Abend et al., Classification of Binary Random Patterns, IEEE Trans. on Information Theory IT-11, 538–544 (Oct. 1965)Google Scholar
  2. 2.
    N. Abramson, “Information Theory and Coding,” McGraw-Hill Book Co., New York (1963)Google Scholar
  3. 3.
    N. Abramson and D. Braverman, Learning to Recognize Patterns in a Random Environment, IRE Trans. on Information Theory IT-8 (5), 558–563 (Sept. 1962)Google Scholar
  4. 4.
    M. A. Aizerman et al., The Robbins-Monro Process and the Method of Potential Functions, Automatics and Telemechanics 26, 1951–1954 (1965)Google Scholar
  5. 5.
    M. A. Aizerman et al., Theoretical Foundations of the Potential Function Method in Pattern Recognition, Automatics and Telemechanics 25, 917–936 (1964)Google Scholar
  6. 6.
    M. A. Aizerman et al., The Probability Problem of Pattern Recognition Learning and the Method of Potential Functions, Automatics and Telemechanics 25, 1307–1323 (1964)Google Scholar
  7. 7.
    M. A. Aizerman et al., The Method of Potential Functions for the Problem of Restoring the Characteristic of a Function Converter from Randomly Observed Points, Automatics and Telemechanics 25, 1705–1714 (1964)Google Scholar
  8. 8.
    A. Albert, A Mathematical Theory of Pattern Recognition, Ann. Math. Stat. 34, 284–299 (March 1963)CrossRefGoogle Scholar
  9. 9.
    L. Alt, Digital Pattern Recognition by Moments, in: “Optical Character Recognition” (G. L. Fisher et al., eds.), pp. 153–179, Spartan Books, Washington, D. C. (1962)Google Scholar
  10. 10.
    S. Amari, A Theory of Adaptive Pattern Classifiers, IEEE Trans. on Electronic Computers EC-16, 299–307 (June 1967)Google Scholar
  11. 11.
    T. W. Anderson and R. R. Bahadur, Classification into Two Multivariate Normal Distributions with Different Covariance Matrices, Ann. Math. Stat. 33 (1962)Google Scholar
  12. 12.
    T. W. Anderson, “Introduction to Multivariate Statistical Analysis,” John Wiley and Sons, New York (1958)Google Scholar
  13. 13.
    O. A. Bashkirov, E. M. Braverman and I. B. Muchnik, Potential Function Algorithms for Pattern Recognition Learning Machine, Automatics and Telemechanics 25, 692–695 (1964)Google Scholar
  14. 14.
    C. Blaydon, On a Pattern Classification Result of Aizerman, Braverman, and Rozonoer, IEEE Trans. on Information Theory IT-12, 83–83 (Jan. 1962)Google Scholar
  15. 15.
    W. W. Bledsoe, Further Results on the N-tuple Pattern Recognition Method, IRE Trans. on Electronic Computers EC-10 (1) p. 96 (March 1961)Google Scholar
  16. 16.
    W. W. Bledsoe and I. Browning, Pattern Recognition and Reading by Machine, in: “Proc. of the Eastern Joint Computer Conference” (1959), AIEEGoogle Scholar
  17. 17.
    H. D. Block, N. J. Nilsson, and R. O. Duda, Determination and Detection of Features in Pattern, in “Computer and Information Sciences-I” (J. T. Tou and R. H. Wilcox, eds.), pp. 75–110, Spartan Books, Washington, D. C. (1964)Google Scholar
  18. 18.
    J. A. Blum, Multidimensional Stochastic Approximation Procedures, Ann. Math. Stat. 6 (4) (1955)Google Scholar
  19. 19.
    E. M. Braverman and E. S. Pyatnitskii, Estimation of the Rate of Convergence of Algorithms Based on the Potential Function Method, Automatics and Telemechanics 27, 95–112 (1966)Google Scholar
  20. 20.
    E. M. Braverman, On the Method of Potential Functions, Automatics and Telemechanics 26, 2205–2213 (1965)Google Scholar
  21. 21.
    D. Braverman, Learning Filters for Optimum Pattern Recognition, IRE Trans. on Information Theory IT-8, 280–285 (July 1962)CrossRefGoogle Scholar
  22. 22.
    D. B. Brick and J. Owen, A Mathematical Approach to Pattern Recognition and Self-Organization, in “Computer and Information Sciences-I” (J. T. Tou and R. H. Wilcox, eds.), pp. 137–168, Spartan Books, Washington, D. C. (1964)Google Scholar
  23. 23.
    D. T. Brown, A Note on Approximations to Discrete Probability Distributions, Information and Control 2, 386–392 (Dec. 1959)CrossRefGoogle Scholar
  24. 24.
    J. A. Bradshaw, Letter Recognition Using a Captive Scan, IEEE Trans. Electronic Computers EC-12, p. 26 (Feb. 1963)Google Scholar
  25. 25.
    W. G. Chaplin and V. S. Levadi, A Generalization of the Linear Threshold Decision Algorithm to Multiple Classes, in “Computer and Information Sciences-II” (J. T. Tou, ed.), pp. 57–59, Academic Press, New York (1967)Google Scholar
  26. 26.
    Y. T. Chien and K. S. Fu, On Bayesian Learning and Stochastic Approximation, IEEE Trans. on Systems Science and Cybernetics SSC-3, 28–38 (June 1967)CrossRefGoogle Scholar
  27. 27.
    27.Y. T. Chien and K. S. Fu, A Modified Sequential Recognition Machine Using Time-Varying Stopping Boundaries, IEEE Trans. on Information Theory IT-12 (2), 206–214 (April 1966)CrossRefGoogle Scholar
  28. 28.
    C. K. Chow and C. N. Lin, An Approach to Structure Adaptation in Pattern Recognition, IEEE Trans. on Systems Science and Cybernetics SCC-2, 73–80 (Dec. 1966)Google Scholar
  29. 29.
    C. K. Chow, A Recognition Method Using Neighbor Dependence, IRE Trans. on Electronic Computers EC-11, 683–690 (Oct. 1962)Google Scholar
  30. 30.
    C. K. Chow, An Optimum Character Recognition System Using Decision Functions, IRE Trans. on Electronic Computers EC-6, 247–254 (Dec. 1957)Google Scholar
  31. 31.
    D. B. Cooper and P. W. Cooper, Nonsupervised Adaptive Signal Detection and Pattern Recognition, Information and Control 7, 416–444 (1964)CrossRefGoogle Scholar
  32. 32.
    P. W. Cooper, Hyperplanes, Hyperspheres, and Hyperquadrics, as Decision Boundaries, in “Computer and Information Sciences-I” (J. T. Tou and R. H. Wilcox, eds.), pp. 111–138, Spartan Books, Washington, D. C. (1964)Google Scholar
  33. 33.
    P. W. Cooper, Some Topics in Nonsupervised Adaptive Detection for Multivariate Normal Distribution, in “Computer and Information Sciences-II” (J. T. Tou, ed.), pp. 123–146, Academic Press, New York (1967)Google Scholar
  34. 34.
    T. M. Cover and P. E. Hart, Nearest Neighbor Pattern Classification, IEEE Trans. on Information Theory IT-13 21–27 (Jan. 1967)CrossRefGoogle Scholar
  35. 35.
    T. M. Cover, Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition, IEEE Trans. on Electronic Computers EC-14, 326–334 (June 1965)Google Scholar
  36. 36.
    W. E. Dickinson, A Character-Recognition Study, IBM J. Res. Dev. 4 (3) (July 1960)CrossRefGoogle Scholar
  37. 37.
    G. P. Dinneen, Programming Pattern Recognition, in: “Proc. of the Western Joint Computer Conference” (1955), AIEEGoogle Scholar
  38. 38.
    R. O. Duda and H. Fossum, Pattern Classification by Iteratively Determined Linear and Piecewise Linear Discriminant Functions, IEEE Trans. on Electronic Computers EC-15, 220–232 (Apr. 1966)Google Scholar
  39. 39.
    A. Dvoretzky, On Stochastic Approximation, in “Proc. of the Third Berkeley Symposium on Mathematical Statistics and Probability,” Vol. 1 (1956)Google Scholar
  40. 40.
    M. Eden, Handwriting and Pattern Recognition, IRE Trans. on Information Theory IT-8, 160–171 (Feb. 1962)CrossRefGoogle Scholar
  41. 41.
    E. A. Feigenbaum and J. Feldman (eds.), “Computers and Thought,” McGraw-Hill Book Co., New York (1963)Google Scholar
  42. 42.
    G. L. Fisher et al. (eds.), “Optical Character Recognition,” Spartan Books, Washington, D. C. (1962)Google Scholar
  43. 43.
    S. C. Fralick, Learning to Recognize a Pattern without a Teacher, IEEE Trans. on Information Theory IT-13, 57–64 (Jan. 1967)CrossRefGoogle Scholar
  44. 44.
    K. S. Fu et al., A Dynamic Programming Approach to Sequential Pattern Recognition, IEEE Trans. on Electronic Computers EC-16, 790–803 (Dec. 1967)Google Scholar
  45. 45.
    K. S. Fu, A Sequential Decision Model for Optimum Recognition, in “Biological Prototypes and Synthetic Systems,” Vol. I (E. E. Bernard and M. R. Kare, eds.), Plenum Press, New York (1962)Google Scholar
  46. 46.
    A. Gamba, New Developments in Artificial Intelligence and Pattern Recognition, in “Computer and Information Sciences-I” (J. T. Tou and R. H. Wilcox, eds.), pp. 219–229, Spartan Books, Washington, D. C. (1964)Google Scholar
  47. 47.
    V. E. Giuliano et al., Automatic Pattern Recognition by a Gestalt Method, Information and Control 4 (4) (Dec. 1961)CrossRefGoogle Scholar
  48. 48.
    E. C. Greanias et al., The Recognition of Handwritten Numerals by Contour Analysis, IBM J. Res. Dev. 7, 14–21 (Jan. 1963)CrossRefGoogle Scholar
  49. 49.
    J. S. Griffin et al., A Pattern Identification Device Using Linear Decision Functions, in “Computer and Information Sciences-I” (J. T. Tou and R. H. Wilcox, eds.), pp. 167–193, Spartan Books, Washington, D. C. (1964)Google Scholar
  50. 50.
    R. L. Grimsdale et al., A System for the Automatic Recognition of Patterns, Proc. IEEE (London) 106, Part B, No. 26, 210–221 (March 1959)Google Scholar
  51. 51.
    J. K. Hawkins, Self-organizing Systems-A Review and Commentary, Proc. IRE 49, 31–48 (Jan. 1961)CrossRefGoogle Scholar
  52. 52.
    W. H. Highleyman, Linear Decision Functions, with Application to Pattern Recognition, Proc. IRE 50, 1501–1514 (June 1962)CrossRefGoogle Scholar
  53. 53.
    W. S. Holmes, H. R. Leland, and G. E. Richmond, Design of a Photo Interpretation Automaton, “Proc. of the Fall Joint Computer Conference,” Vol. 22 (1962), AFIP, The National Press, Palo Alto, CalifGoogle Scholar
  54. 54.
    L. P. Horowitz and G. L. Shelton, Pattern Recognition Using Autocorrelation, Proc. IRE Jan. 1961Google Scholar
  55. 55.
    L. A. Kamentsky and C. N. Liu, A Theoretical and Experimental Study of a Model for Pattern Recognition, in “Computer and Information Sciences-I” (J. T. Tou and R. H. Wilcox, eds.), pp. 194–218, Spartan Books, Washington, D. C. (1964)Google Scholar
  56. 56.
    L. A. Kamentsky and C. N. Liu, Computer-Automated Design of Multifont Print Recognition Logic, IBM J. Res. Dev. 7, 2–13 (Jan. 1963)CrossRefGoogle Scholar
  57. 57.
    L. A. Kamentsky, Pattern and Character Recognition Systems-Picture Processing by Nets of Neuron-Like Elements, in “Proc. of the Western Joint Computer Conference,” (1959), pp. 304–309, AIEEGoogle Scholar
  58. 58.
    L. N. Kanal and N. C. Randall, “Recognition System Design by Statistical Analysis, in “Proc. of the 19th National Conference,” ACM Publication P-64, Aug. 1964Google Scholar
  59. 59.
    L. Kanal, “Evaluation of a Class of Pattern Recognition Networks, in “Biological Prototypes and Synthetic Systems,” Vol. I, (E. E. Bernard and M. R. Kare, eds.), pp. 261–269, Plenum Press, New York (1962)Google Scholar
  60. 60.
    H. Kazmierczak and K. Steinbuch, Adaptive Systems in Pattern Recognition, IEEE Trans. on Electronic Computers EC-12, 822–835 (Dec. 1963)Google Scholar
  61. 61.
    D. G. Keehn, A Note on Learning for Gaussian Properties, IEEE Trans. on Information Theory Jan. 1965Google Scholar
  62. 62.
    J. Kiefer and J. Wolfowitz, Stochastic Estimation of the Maximum of a Regression Function, Ann. Math. Stat. 23 (3) (1952)Google Scholar
  63. 63.
    J. S. Koford and G. F. Groner, The Use of an Adaptive Threshold Element to Design a Linear Optimal Pattern Classifier, IEEE Trans. on Information Theory IT-12, 42–50 (Jan. 1966)CrossRefGoogle Scholar
  64. 64.
    V. A. Kovalevsky, Present and Future of Pattern Recognition Theory, in “Proc. IFIP Congress 65,” Vol. I, pp. 37–43, Spartan Books, Washington D. C. (1966)Google Scholar
  65. 65.
    L. S. G. Kovasznay, and H. M. Joseph, Image Processing, Proc. IRE, 43, 560–570 (Mar 1955)CrossRefGoogle Scholar
  66. 66.
    J. J. Leimer, Design Factors in the Development of an Optical Character Recognition System, IRE Trans. on Information Theory IT-8, 161–171 (Feb. 1962)CrossRefGoogle Scholar
  67. 67.
    P. M. Lewis, The Characteristic Selection Problem in Recognition Systems, IRE Trans. on Information Theory IT-8, 171–178 (Feb. 1962)CrossRefGoogle Scholar
  68. 68.
    P. M. Lewis, Approximating Probability Distributions to Reduce Storage Requirements, Information and Control 2, 214–225 (1959)CrossRefGoogle Scholar
  69. 69.
    C. N. Liu, A Programmed Algorithm for Designing Multifont Character Recognition Logic, IEEE Trans. on Electronic Computers EC-13, 586–593 (Oct. 1964)CrossRefGoogle Scholar
  70. 70.
    N. V. Loginov, Methods of Stochastic Approximation, Automatics and Telemechanics 27, 185–204 (1966)Google Scholar
  71. 71.
    T. Marill and D. M. Green, On the Effectiveness of Receptors in Recognition Systems, IEEE Trans. on Information Theory IT-9, 11–17 (Jan. 1963)CrossRefGoogle Scholar
  72. 72.
    T. Marill, Automatic Recognition of Speech, IRE Trans. on Human Factors Electronics HFE-2, 34–38 (Mar. 1961)CrossRefGoogle Scholar
  73. 73.
    T. Marill and D. M. Green, Statistical Recognition Functions and the Design of Pattern Recognizers, IRE Trans. on Electronic Computers EC-9, 472–477 (Dec. 1960)Google Scholar
  74. 74.
    R. L. Mattson and J. E. Damman, A Technique for Determining and Coding Subclasses in Pattern Recognition Problems, IBM J. Res. Dev. 9 (4) (July 1965)CrossRefGoogle Scholar
  75. 75.
    M. Minsky, Steps Toward Artificial Intelligence, Proc. IRE 49, 8–30 (Jan. 1961)CrossRefGoogle Scholar
  76. 76.
    M. Nadler, An Analog-Digital Character Recognition System, IEEE Trans. on Electronic Computers EC-12, 814–821 (Dec. 1963)Google Scholar
  77. 77.
    Z. J. Nikolic and K. S. Fu, An Algorithm for Learning Without External Supervision and Its Application to Learning Control Systems, IEEE Trans. on Automatic Control AC-11, 414–422 (July 1966)CrossRefGoogle Scholar
  78. 78.
    N. J. Nilsson, “Learning Machines,” McGraw-Hill Book Co., New York (1965)Google Scholar
  79. 79.
    A. B. J. Novikoff, Integral Geometry as a Tool in Pattern Perception, in “Principles of Self-Organization,” (H. Von Foerster and G. W. Zopf, Jr., eds.), Pergamon Press, New York, (1962)Google Scholar
  80. 80.
    F. M. Reza, “An Introduction to Information Theory,” McGraw-Hill Book Co., New York (1961)Google Scholar
  81. 81.
    H. Robbins, and S. Monro, A Stochastic Approximation Method, Ann. Math. Stat. 22 (1) (1951)CrossRefGoogle Scholar
  82. 82.
    F. Rosenblatt, “Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms,” Spartan Books, Washington, D. C. (1961)Google Scholar
  83. 83.
    A. Rosenfeld, An Approach to Automatic Photographic Interpretation, Photogrammetric Engineering 27, pp. 660–665 (Sept. 1962)Google Scholar
  84. 84.
    G. S. Sebestyen, and J. Edie, An Algorithm for Nonparametric Pattern Recognition, IEEE Trans. on Electronic Computers EC-15, 908–915 (Dec. 1966)Google Scholar
  85. 85.
    G. S. Sebestyen, Pattern Recognition by an Adaptive Process of Sample Set Construction, IRE Trans. on Information Theory IT-8, 582–591 (Sept. 1962)Google Scholar
  86. 86.
    G. S. Sebestyen, “Decision Making Processes in Pattern Recognition,” The Macmillan Company, New York (1962)Google Scholar
  87. 87.
    O. G. Selfridge, Pattern Recognition and Learning, in “Information Theory” (C. Cherry, ed.), Butterworth Scientific Publications, London (1956)Google Scholar
  88. 88.
    H. Sherman, A Quasi-Topological Method for Machine Recognition of Line Patterns, in “Proc. International Conference on Information Processing,” Paris, 1959, Butterworth Scientific Publications, London (1960)Google Scholar
  89. 89.
    D. F. Specht, Generation of Polynomial Discriminant Functions for Pattern Recognition, IEEE Trans. on Electronic Computers EC-16, 308–319 (June 1967)Google Scholar
  90. 90.
    R. J. Spinrad, Machine Recognition of Hand Printing, Information and Control 8, 124–142 (Apr. 1965)CrossRefGoogle Scholar
  91. 91.
    J. Spragins, A Note on the Iterative Application of Bayes’ Rule, IEEE Trans. on Information Theory IT-11, 544–549 (1965)CrossRefGoogle Scholar
  92. 92.
    W. Sprick and K. Ganzhorn, An Analogous Method for Pattern Recognition by Following the Boundary, in “Proceedings of the International Conference on Information Processing,” Paris, 1959, Butterworth Scientific Publications, London (1960)Google Scholar
  93. 93.
    S. D. Stearns, A Method for the Design of Pattern Recognition Logic, IRE Trans. on Electronic Computers, EC-9, 48–53 (March 1960)Google Scholar
  94. 94.
    G. P. Steck, Stochastic Model for the Browning-Bledsoe Pattern Recognition Scheme, IRE Trans. on Electronic Computers EC-11, 274–282 (Apr. 1962)Google Scholar
  95. 95.
    D. M. Stern and D. W. C. Shen, Character Recognition by Context-Dependent Transformations, Proc. IEEE (British) III (11) (Nov. 1964)Google Scholar
  96. 96.
    Ya. Z. Tsypkin, Optimization, Adaptation, and Learning in Automatic System, in “Computer and Information Sciences-II” (J. T. Tou, ed.), pp. 15–31, Academic Press, New York (1967)Google Scholar
  97. 97.
    Ya. Z. Tsypkin, Establishing Characteristics of a Function Transformer from Randomly Observed Points, Automatics and Telemechanics 26, 1947–1950 (1965)Google Scholar
  98. 98.
    J. T. Tou, Feature Extraction in Pattern Recognition, in “Pattern Recognition,” Vol. I, Pergamon Press, New York (1968)Google Scholar
  99. 99.
    J. T. Tou, Information Theoretic Approach to Pattern Recognition, IEEE International Convention Record (1968)Google Scholar
  100. 100.
    J. T. Tou, (ed.) “Computer and Information Scienees-II,” Academic Press, New York (1967)Google Scholar
  101. 101.
    J. T. Tou and R. P. Heydorn, Some Approaches to Optimum Feature Extraction, in “Computer and Information Sciences-II” (J. T. Tou, ed.), pp. 57–59, Academic Press, New York (1967)Google Scholar
  102. 102.
    J. T. Tou and R. H. Wilcox (eds.), “Computer and Information Sciences-I,” Spartan Books, Washington, D. C. (1964)Google Scholar
  103. 103.
    L. Uhr, “Pattern Recognition,” John Wiley and Sons, New York, (1966)Google Scholar
  104. 104.
    S. H. Unger, Pattern Detection and Recognition, Proc. IRE 47, 1737–1752Google Scholar
  105. 105.
    S. Watanabe et al., Evaluation and Selection of Variables in Pattern Recognition, in “Computer and Information Sciences-II” (J. T. Tou, ed.), pp. 92–121, Academic Press, New YorkGoogle Scholar
  106. 106.
    S. Watanabe (ed.), “Methodologies in Pattern Recognition,” Academic Press, New YorkGoogle Scholar
  107. 107.
    B. Widrow, Generalization and Information Storage in Networks of Adaline Neurons, in “Self-Organizing Systems” (M. C. Yovits et al., eds.) Spartan Books, Washington, D. C. (1962)Google Scholar
  108. 108.
    R. O. Winder, The Fundamentals of Threshold Logic, in “Applied Automata Theory” (J. T. Tou, ed.), Academic Press, New York (1968)Google Scholar
  109. 109.
    M. C. Yovits et al. (eds.), “Self-Organizing Systems,” Spartan Books, Washington, D. C. (1962)Google Scholar

Copyright information

© Plenum Press 1969

Authors and Affiliations

  1. 1.College of EngineeringUniversity of FloridaGainesvilleUSA

Personalised recommendations