Linear Classifiers for Nonseparable Classes

  • Jack Sklansky
  • Gustav N. Wassel


In this chapter we describe several training procedures based on the gradient and stochastic approximation techniques of the preceding chapter. Although our discussion in this chapter is restricted to two-class cases, the techniques may be extended to multiple-class cases by using the concepts of Section 2.6. The training procedures of this chapter apply to pairs of classes that are linearly nonseparable, i.e., that are not linearly separable, as well as those that are linearly separable. Linearly nonseparable classes include pairs of classes that are separable but not by a hyperplane, as well as pairs of classes that overlap.


Feature Vector Weight Vector Training Procedure Initial Vector Linear Classifier 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    G. N. Wassel, Training a linear classifier to optimize the error probability. Ph.D. Dissertation, UCI Pattern Recognition Project, School of Engineering, University of California, Irvine, California, Technical Report TP-72–5, December 1972.Google Scholar
  2. 2.
    R. O. Duda and P. E. Hart, Pattern Recognition and Scene Analysis. John Wiley & Sons, New York, 1973.Google Scholar
  3. 3.
    G. N. Wassel and J. Sklansky, Training a one-dimensional classifier to minimize the probability of error. IEEE Trans, on Systems, Man, and Cybernetics, SMC-2: 533–541 (1972).MathSciNetCrossRefGoogle Scholar
  4. 4.
    J. Kiefer and J. Wolfowitz, Stochastic estimation of the maximum of a regression function. The Annals of Mathematical Statistics, 23: 462–466 (1952).MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    A. Dvoretzky, On stochastic approximation. In: Proc. Third Berkeley Symp. Mathematical Statistics and Probability, December 1954, June-July 1955, pp. 39–55.Google Scholar
  6. 6.
    J. R. Blum, Multidimensional stochastic approximation methods. The Annals of Mathematical Statistics, 25, (4): 1954.Google Scholar
  7. 7.
    C. Blaydon, Recursive algorithms for pattern classification. Division of Engineering and Applied Physics, Harvard University, Cambridge, Massachusetts, Technical Report 520, March 1967.Google Scholar
  8. 8.
    C. C. Blaydon and Y. C. Ho, Recursive algorithms for pattern classification. Proceedings of 1966 National Electronics Conference.Google Scholar
  9. 9.
    Y. C. Ho and R. L. Kashyap, An algorithm for linear inequalities and its applications. IEEE Trans, on Electronic Computers, EC-14: 683–688 (1965).CrossRefGoogle Scholar
  10. 10.
    G. N. Wassel and J. Sklansky, An adaptive nonparametric linear classifier. Proc. of the IEEE, 64, (8): 1162–1171 (1976).MathSciNetCrossRefGoogle Scholar
  11. 11.
    K. Fukunaga, Introduction to Statistical Pattern Recognition. Academic Press, New York, 1972.Google Scholar
  12. 12.
    B. Widrow and M. E. Hoff, Adaptive switching circuits. 1960 IRE WESCON Convention Record, Part 4, August 1960, pp. 96–104.Google Scholar
  13. 13.
    W. C. Miller, A modified mean-square-error criterion for use in unsupervised learning. Technical Report No. 6778–2, Stanford University, August 1967.Google Scholar
  14. 14.
    H. Do-Tu and M. Installe, Learning algorithms for nonparametric solution to the minimum error classification problem. IEEE Trans, on Computers, C-27: 648–659 (1978).MathSciNetCrossRefGoogle Scholar
  15. 15.
    C. Kimme-Smith, G. Wassel, G. Frankl, and J. Sklansky, Toward a computerized estimate of the index of risk for developing breast cancer. UCI Technical Report TP-79–3, University of California, School of Engineering, Irvine, California, 1979.Google Scholar
  16. 16.
    C. Hightower, K. Rowe, and J. Sklansky, Target pattern correlation for BMD threat discrimination. Proceedings of the Twenty-Third Tri-Service Radar Symposium, West Point, New York, July 1977.Google Scholar
  17. 17.
    J. N. Wolfe, Breast pattern as an index of risk for development breast cancer. American Journal of Roentgenology, 126: 1130–1139 (1976).Google Scholar
  18. 18.
    M. J. Lighthill, Introduction to Fourier Analysis and Generalized Functions. Cambridge University Press, New York, 1958.Google Scholar
  19. 19.
    R. A. Fisher, The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7 (Part II): 179–188 (1936). Reprinted in: Contributions to Mathematical Statistics, John Wiley & Sons, New York, 1950Google Scholar

Copyright information

© Springer-Verlag New York Inc 1981

Authors and Affiliations

  • Jack Sklansky
    • 1
  • Gustav N. Wassel
    • 2
  1. 1.Department of Electrical EngineeringUniversity of California at IrvineIrvineUSA
  2. 2.Deparment of Electronic and Electrical EngineeringCalifornia Polytechnic State UniversitySan Luis ObispoUSA

Personalised recommendations