Skip to main content
SpringerLink
Log in

Basic Concepts of Probabilistic Neural Networks

  • Chapter
  • First Online:
Stream Data Mining: Algorithms and Their Probabilistic Properties

Part of the book series: Studies in Big Data ((SBD,volume 56))

  • 969 Accesses

Abstract

Probabilistic neural networks (PNN), introduced by Specht [1, 2] have their predecessors in the theory of statistical pattern classification. In the fifties and sixties, problems of statistical pattern classification in the stationary case were accomplished by means of parametric methods, using the available apparatus of statistical mathematics (e.g. [3,4,5,6,7]). The knowledge of the probability density to an accuracy of unknown parameters was assumed and the parameters were estimated based on the learning sequence. Having observed tendencies present in the literature within the next decades, we should say that these methods have been almost completely replaced by the non-parametric approach (see e.g. [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24]). In the non-parametric approach it is assumed that a functional form of probability densities is unknown.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Specht, D.: Probabilistic neural networks and the polynomial Adaline as complementary techniques for classification. IEEE Trans. Neural Netw. 1, 111–121 (1990)

    Article  Google Scholar 

  2. Specht, D.F.: A general regression neural network. IEEE Trans. Neural Netw. 2(6), 568–576 (1991)

    Article  Google Scholar 

  3. Bishop, C.: Neural Networks for Pattern Recognition. Clarendon Press, Oxford (1995)

    MATH  Google Scholar 

  4. Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley, London (2001)

    Google Scholar 

  5. Fu, K.: Sequential Methods in Pattern Recognition and Machine Learning. Academic, New York (1968)

    MATH  Google Scholar 

  6. Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic, New York (1990)

    MATH  Google Scholar 

  7. Webb, A.: Statistical Pattern Recognition. Wiley, Chichester (2002)

    Google Scholar 

  8. Devroye, L., Györfi, L.: Nonparametric Density Estimation: The \(L_1\) View. Wiley, New York (1985)

    Google Scholar 

  9. Devroye, L., Györfi, L., Lugosi, G.: Probabilistic Theory of Pattern Recognition. Springer, New York (1996)

    Google Scholar 

  10. Devroye, L., Lugosi, G.: Combinatorial Methods in Density Estimation. Springer, New York (2001)

    Google Scholar 

  11. Efromovich, S.: Nonparametric Curve Estimation. Methods, Theory and Applications. Springer, New York (1999)

    Google Scholar 

  12. Eubank, R.L.: Spline Smoothing and Nonparametric Regression. Marcel Dekker, INC., New York (1988)

    Google Scholar 

  13. Eubank, R.: Nonparametric Regression and Spline Smoothing. Marcel Dekker, New York (1999)

    MATH  Google Scholar 

  14. Györfi, L., Hżrdle, W., Sarda, P., Vieu, P.: Nonparametric Curve Estimation from Time Series. Springer, New York (1989)

    Book  Google Scholar 

  15. Györfi, L., Kohler, M., Krzyżak, A., Walk, H.: A Distribution-Free Theory of Nonparametric Regression. Springer, New York (2002)

    Book  Google Scholar 

  16. Härdle, W.: Applied Nonparametric Regression. Cambridge University Press, Cambridge (1990)

    Google Scholar 

  17. Härdle, W., Kerkyacharian, G., Picard, D., Tsybakov, A.: Wavelets, Approximation, and Statistical Applications. Springer, New York (1998)

    Book  Google Scholar 

  18. Ibragimov, I., Khasminskii, R.: Statistical Estimation: Asymptotic Theory. Springer, New York (1981)

    Book  Google Scholar 

  19. Pagan, A., Ullah, A.: Nonparametric Econometrics. Cambridge University Press, London (1999)

    Book  Google Scholar 

  20. Rafajłowicz, E.: Consistency of orthogonal series density estimators based on grouped observations. IEEE Trans. Inf. Theory 43(1), 283–285 (1997)

    Article  MathSciNet  Google Scholar 

  21. Rao, B.L.S.P.: Nonparametric Functional Estimatio. Academic, New York (1983)

    Google Scholar 

  22. Thompson, J., Tapia, R.: Nonparametric Function Estimation and Simulation. SIAM, Philadelphia (1990)

    Google Scholar 

  23. Wertz, W.: Statistical Density Estimation: a Survey. Vandenhoeck & Ruprecht, Göttingen (1978)

    Google Scholar 

  24. Wertz, W., Schneider, B.: Statistical density estimation: a bibliography. Int. Stat. Rev. 47, 155–175 (1979)

    MathSciNet  MATH  Google Scholar 

  25. Rosenblatt, M.: Remarks on some estimates of a density function. Ann. Math. Stat. 27, 155–175 (1956)

    MathSciNet  Google Scholar 

  26. Parzen, E.: On estimation of probability density function and mode. Ann. Math. Stat. 33, 1065–1076 (1962)

    Article  MathSciNet  Google Scholar 

  27. Cacoullos, T.: Estimation of a multivariate density. Ann. Inst. Stat. Math. 18, 179–189 (1965)

    Article  MathSciNet  Google Scholar 

  28. Čencov, N.: Evaluation of an unknown distribution density from observations. Sov. Math. 3, 1559–1562 (1962)

    Google Scholar 

  29. Schwartz, S.: Estimation of probability density by an orthogonal series. Ann. Math. Stat. 1261–1265 (1967)

    Google Scholar 

  30. Kronmal, R., Tarter, M.: The estimation of probability densities and cumulatives by Fourier series methods. J. Am. Stat. Assoc. (1968)

    Google Scholar 

  31. Walter, G.: Properties of Hermite series estimation of probability density. Ann. Stat. 5, 1258–1264 (1977)

    Article  MathSciNet  Google Scholar 

  32. Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13, 21–27 (1967)

    Article  Google Scholar 

  33. Loftsgaarden, D., Quesenberry, C.: A nonparametric estimate of a multivariate density function. Ann. Math. Stat. 36, 1049–1051 (1965)

    Article  MathSciNet  Google Scholar 

  34. Devroye, L.: Universal consistency in nonparametric regression and nonparametric discrimination. Technical report. School of Computer Science, Mc Gill University (1978)

    Google Scholar 

  35. Greblicki, W.: Asymptotically optimal pattern recognition procedures with density estimate. IEEE Trans. Inf. Theory 24, 250–251 (1978)

    Article  MathSciNet  Google Scholar 

  36. Greblicki, W., Rutkowski, L.: Density-free Bayes risk consistency of nonparametric pattern recognition procedures. Proc. IEEE 69(4), 482–483 (1981)

    Article  Google Scholar 

  37. Rutkowski, L.: Sequential estimates of probability densities by orthogonal series and their application in pattern classification. IEEE Trans. Syst. Man Cybern. SMC-10(12), 918–920 (1980)

    Google Scholar 

  38. Rutkowski, L.: Sequential estimates of a regression function by orthogonal series with applications in discrimination. Lectures Notes in Statistics, vol. 8, pp. 236–244. Springer, New York (1981)

    Google Scholar 

  39. Rutkowski, L.: Sequential pattern recognition procedures derived from multiple Fourier series. Pattern Recognit. Lett. 8, 213–216 (1988)

    Article  Google Scholar 

  40. Ryzin, J.: Bayes risk consistency of classification procedures using density estimation. Sankhya Ser. A (1966)

    Google Scholar 

  41. Wolverton, C., Wagner, T.: Asymptotically optimal discriminant functions for pattern classification. IEEE Trans. Inf. Theory 15, 258–265 (1969)

    Article  MathSciNet  Google Scholar 

  42. Kramer, C., Mckay, B., Belina, J.: Probabilistic neural network array architecture for ECG classification. In: Proceedings of the Annual International Conference on IEEE Engineering in Medicine and Biology Society, vol. 17, pp. 807–808 (1995)

    Google Scholar 

  43. Musavi, M., Chan, K., Hummels, D., Kalantri, K.: On the generalization ability of neural-network classifier. IEEE Trans. Pattern Anal. Mach. Intell. 16, 659–663 (1994)

    Article  Google Scholar 

  44. Raghu, P., Yegnanarayana, B.: Supervised texture classification using a probabilistic neural network and constraint satisfaction model. IEEE Trans. Neural Netw. 9, 516–522 (1998)

    Article  Google Scholar 

  45. Romero, R., Touretzky, D., Thibadeau, G.: Optical Chinese character recognition using probabilistic neural networks. Pattern Recognit. 3, 1279–1292 (1997)

    Article  Google Scholar 

  46. Streit, R.L., Luginbuhl, T.: Maximum likelihood training of probabilistic neural networks. IEEE Trans. Neural Netw. 5, 764–783 (1994)

    Article  Google Scholar 

  47. Burrascano, P.: Learning vector quantization for the probabilistic neural network. IEEE Trans. Neural Netw. 2, 458–461 (1991)

    Article  Google Scholar 

  48. Zaknich, A.: A vector quantization reduction method for the probabilistic neural network. In: Proceedings of the IEEE International Conference on Neural Networks: Piscataway, NJ, pp. 1117–1120 (1997)

    Google Scholar 

  49. Specht, D.: Enhancements to the probabilistic neural networks. In: Proceedings of the IEEE International Joint Conference on Neural Networks: Baltimore, MD, pp. 761–768 (1992)

    Google Scholar 

  50. Mao, K., Tan, K.C., Ser, W.: Probabilistic neural-network structure determination for pattern classification. IEEE Trans. Neural Netw. 11(4), 501–507 (2000)

    Article  Google Scholar 

  51. Jones, M., Marron, J., Sheather, S.: A brief survey of bandwidth selection for density estimation. J. Am. Stat. Assoc. 91, 401–407 (1996)

    Article  MathSciNet  Google Scholar 

  52. Földes, A., Révész, P.: A general method for density estimation. Stud. Sci. Math. Hung. 9, 81–92 (1974)

    MathSciNet  MATH  Google Scholar 

  53. Nikolsky, S.: A Course of Mathematical Analysis. Mir Publishers, Moscow (1977)

    Google Scholar 

  54. Szegö, G.: Orthogonal Polynomials, vol. 23. American Mathematical Society, Colloquium Publications (1959)

    Google Scholar 

  55. Sansone, G.: Orthogonal Functions. Interscience Publishers Inc., New York (1959)

    MATH  Google Scholar 

  56. Alexits, G.: Convergence Problems of Orthogonal Series. Akademiai Kiado, Hungary, Budapest (1961)

    Google Scholar 

  57. Zygmund, A.: Trigonometric Series. Cambridge University Press, Cambridge (1959)

    MATH  Google Scholar 

  58. Sjölin, P.: Convergence almost everywhere of certain singular integrals and multiple Fourier series. Ark. Math. 9, 65–90 (1971)

    Article  MathSciNet  Google Scholar 

  59. Yamato, H.: Sequential estimation of a continuous probability density function and the mode. Bull. Math. Stat. 14, 1–12 (1971)

    Article  MathSciNet  Google Scholar 

  60. Nadaraya, E.A.: On estimating regression. Theory Probab. Appl. 9(1), 141–142 (1964)

    Article  Google Scholar 

  61. Watson, G.S.: Smooth regression analysis. Sankhyā: Indian J. Stat. Ser. A 359–372 (1964)

    Google Scholar 

  62. Devroye, L., Wagner, T.: On the convergence of kernel estimators of regression functions with applications in discrimination. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 51, 15–21 (1980)

    Article  MathSciNet  Google Scholar 

  63. Devroye, L.: On the almost everywhere convergence of nonparametric regression function estimates. Ann. Stat. 9, 1301–1309 (1981)

    Article  MathSciNet  Google Scholar 

  64. Devroye, L., Krzyżak, A.: An equivalence theorem for \(l_1\) convergence of the kernel regression estimate. J. Stat. Plan. Inference 23, 71–82 (1989)

    Google Scholar 

  65. Rutkowski, L.: On system identification by nonparametric function fitting. IEEE Trans. Autom. Control AC-27, 225–227 (1982)

    Google Scholar 

  66. Rutkowski, L., Rafajłowicz, E.: On global rate of convergence of some nonparametric identification procedures. IEEE Trans. Autom. Control AC-34(10), 1089–1091 (1989)

    Google Scholar 

  67. Rutkowski, L.: Identification of MISO nonlinear regressions in the presence of a wide class of disturbance. IEEE Trans. Inf. Theory IT-37, 214–216 (1991)

    Google Scholar 

  68. Rutkowski, L.: Multiple Fourier series procedures for extraction of nonlinear regressions from noisy data. IEEE Trans. Signal Process. (1993)

    Google Scholar 

  69. Gałkowski, T., Rutkowski, L.: Nonparametric recovery of multivariate functions with applications to system identification. Proc. IEEE 73, 942–943 (1985)

    Article  Google Scholar 

  70. Gałkowski, T., Rutkowski, L.: Nonparametric fitting of multivariable functions. IEEE Trans. Autom. Control AC-31, 785–787 (1986)

    Google Scholar 

  71. Devroye, L., Wagner, T.: Nonparametric discrimination and density estimation. Technical report 183, Electronic Research Center, University of Texas (1976)

    Google Scholar 

  72. Wahba, G.: Interpolating spline methods for density estimation, variable knots. Technical report 337, Department of Statistics, University of Wisconsin, Madison (1973)

    Google Scholar 

  73. Wahba, G.: Optimal convergence properties of variable knot, kernel, and orthogonal series methods for density estimatio. Ann. Stat. (1975)

    Google Scholar 

  74. Wahba, G.: Smoothing noisy data with spline function. Numer. Math. (1975)

    Google Scholar 

  75. Wahba, G.: Interpolating spline methods for density estimation, equi-spaced knot. Ann. Stat. (1975)

    Google Scholar 

  76. Wahba, G.: A survey of some smoothing problems and the method of generalized cross-validation for solving the, TR-457, Department of Statistics, University of Wisconsin, p. brak (1976)

    Google Scholar 

  77. Wahba, G.: Spline Models for Observational Data. SIAM, Philadelphia (1990)

    Book  Google Scholar 

  78. Devroye, L.: Necessary and sufficient conditions for the almost everywhere convergence of nearest neighbor regression function estimates. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 61, 467–481 (1982)

    Article  MathSciNet  Google Scholar 

  79. Devroye, L., Györfi, L., Krzyżak, A., Lugosi, G.: On the strong universal consistency of nearest neighbor regression function estimates. Ann. Stat. 22, 1371–1385 (1994)

    Article  MathSciNet  Google Scholar 

  80. Aizerman, M., Braverman, E., Rozonoer, L.: Theoretical foundations of the potential function method in pattern recognition learning. Autom. Remote Control 25, 821–837 (1964)

    MATH  Google Scholar 

  81. Chen, S., Cowan, C., Grant, P.: Orthogonal least squares learning algorithm for radial basis network. IEEE Trans. Neural Netw. 2, 302–309 (1991)

    Article  Google Scholar 

  82. Kecman, V.: Learning and Soft Computing. MIT, Cambridge (2001)

    MATH  Google Scholar 

  83. Chui, C.: Wavelets: a Tutorial in Theory and Applications. Academic, Boston (1992)

    MATH  Google Scholar 

  84. Meyer, Y.: Wavelets: Algorithms and Applications. SIAM, Philadelphia (1993)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computational Intelligence, Czestochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski, Maciej Jaworski & Piotr Duda

  2. Information Technology Institute, University of Social Sciences, Lodz, Poland

    Leszek Rutkowski

Authors
  1. Leszek Rutkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maciej Jaworski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Piotr Duda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leszek Rutkowski .

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Rutkowski, L., Jaworski, M., Duda, P. (2020). Basic Concepts of Probabilistic Neural Networks. In: Stream Data Mining: Algorithms and Their Probabilistic Properties. Studies in Big Data, vol 56. Springer, Cham. https://doi.org/10.1007/978-3-030-13962-9_8

Download citation

Publish with us

Policies and ethics

Search

Navigation