Neural Networks Approach

Part of the Advances in Industrial Control book series (AIC)


Neural Network Hide Layer Radial Basis Function Network Probabilistic Neural Network Neural Network Approach 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Aizerman MA, Braverman EM, and Rozenoer LI (1964) Theoretical foundation of potential function method in pattern recognition. Automation and Remote Control 25: 917–936.Google Scholar
  2. [2]
    Akaike H (1970) Statistical predictor identification, Annals of the Institute of Statistical Maths., 22: 202–217.Google Scholar
  3. [3]
    Almeida LB (1987) A learning rule for asynchronous perceptrons with feedback in a combinatorial Environment. IEEE 1st International Conf. on Neural Networks, San Diego, CA II:609–618.Google Scholar
  4. [4]
    Amari S and Maginu K (1988) Statistical neurodynamics of associative memory, Neural Networks 1: 63–73.CrossRefGoogle Scholar
  5. [5]
    Anders U and Korn O (1999) Model selection in neural networks. Neural Networks 12: 309–323.CrossRefGoogle Scholar
  6. [6]
    Bashkirov OA, Braverman EM, and Muchnik IB (1964) Potential function algorithms for pattern recognition learning machines. Automation and Remote Control 25:692–695.Google Scholar
  7. [7]
    Bates JM and Granger CWJ (1969) The combination of forecasts, Operation Research Quart. 20: 451–461.Google Scholar
  8. [8]
    Baum EB and Haussler D (1989) What Size Net Gives Valid Generalisation? Neural Computation 1:151–160.Google Scholar
  9. [9]
    Bethea RM and Rhinehard RR (1991) Applied Engineering Statistics. Marcel Dekker, New York.zbMATHGoogle Scholar
  10. [10]
    Block HD (1962) The Perceptron: a model of brain functioning. Review of Modern Physics, 34:123–135.MathSciNetzbMATHGoogle Scholar
  11. [11]
    Broomhead DS and Lowe D (1988) Multivariable functional interpolation and adaptive networks. Complex Systems 2: 321–355.MathSciNetzbMATHGoogle Scholar
  12. [12]
    Butine WL and Weigend AS (1994) Computing Second Derivatives in Feedforward Networks: A Review. IEEE Trans. on Neural Networks 3: 480–488.Google Scholar
  13. [13]
    Chakraborty K, Mehrotra K, Mohan ChK, Rankas (1992) Forecasting the behavior of Multivariate Time Series Using Neural Networks. Neural Networks 5: 961–970.CrossRefGoogle Scholar
  14. [14]
    Cichocki A and Unbehauen R (1993) Neural Networks for Optimization and Signal Processing. Wiley, Chichester, West Sussex, UK.zbMATHGoogle Scholar
  15. [15]
    Cohen MA and Grossberg S (1983) Absolute Stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. on Systems, Man, and Cybernetics 13: 815–826.MathSciNetzbMATHGoogle Scholar
  16. [16]
    Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Mathematical Control Signals Systems 2:303–314.MathSciNetzbMATHGoogle Scholar
  17. [17]
    Denton JW (1995) How good are neural networks for causal forecasting? J. of Business Forecasting 14(2):17–20.MathSciNetGoogle Scholar
  18. [18]
    Elman JL (1990) Finding structure in time. Cognitive Science 14: 179–211.CrossRefGoogle Scholar
  19. [19]
    Fogel DB (1991) An Information Criterion for Optimal Neural Network Selection. IEEE Trans. On Neural Networks 2:490–499.CrossRefGoogle Scholar
  20. [20]
    Forster WR, Collopy F, Ungar LH (1992) Neural network forecasting of short, noisy time series. Computers and Chemical Engineering16(2): 293–297.Google Scholar
  21. [21]
    German SE, Bienenstock, and Doursat R (1992) Neural networks and the bias/variance dilemma. Neural Computation 1: 1–58.Google Scholar
  22. [22]
    Girosi F and Poggio T (1989) Representation Properties of Networks: Kolmogorov’s Theorem is Irrelevant. Neural Computation 1: 465–469.Google Scholar
  23. [23]
    Girosi F and Poggio T (1990) Networks and the best approximation properties. Biological Cybernetics:169–176.Google Scholar
  24. [24]
    Gorr WL, Nagin D, Szczypula J (1994) Comparative study of artificial neural network and statistical models predicting student grade point averages. Intl. J. of Forecasting 10: 17–34.Google Scholar
  25. [25]
    Grossberg S (1988) Competitive Learning: From interactive activation to adaptive resonance, Neural Networks and Neural Intelligence, Grossberg S. (Eds.), MIT Press, Cambridge, MA.Google Scholar
  26. [26]
    Hagan MT and Menhaj MB (1994) Training feedforward networks with the Marquardt algorithm, IEEE Trans. on Neural Networks, vol. 5(6): 989–993.CrossRefGoogle Scholar
  27. [27]
    Hann TH and Steurer E. (1996) Much ado about nothing? Exchange rate forecasting: Neural networks vs. linear using monthly and weekly data. Neurocomputing 10: 323–339.CrossRefzbMATHGoogle Scholar
  28. [28]
    Hansen IK and Rasmussen CE (1994) Pruning from adaptive regularization. Neural Computation 6: 1223–1232.zbMATHGoogle Scholar
  29. [29]
    Harald PG and Kamastra M (1997) Evolving artificial neural networks to combine the financial forecasts, IEEE Trans. on Evolutionary Computation, vol. 1(1): 40–51.Google Scholar
  30. [30]
    Hassibi B, Stork DG, and Wolff GJ (1992) Optimal brain surgeon and general network pruning. IEEE Intl Conf on Neural Networks, San Francisco 1:293–299.Google Scholar
  31. [31]
    Haykin S (1994) Neural Networks: a comprehensive foundation. McMillan, USAGoogle Scholar
  32. [32]
    Hebb DO (1949) The organisation of behaviour. Wiley, New York.Google Scholar
  33. [33]
    Hecht-Nielsen R (1987a) Counterpropagation Networks. Applied Optics 26(23): 4979–4984.Google Scholar
  34. [34]
    Hecht-Nielsen R (1987b) Kolmogorov’s Mapping Neural Network Existence Theorem, IEEE Conf. On Neural Networks; San Diego, CA. III: 11–14.Google Scholar
  35. [35]
    Hecht-Nielsen R (1988) Application of counterpropagation networks, Neural Networks 1: 131–139.CrossRefGoogle Scholar
  36. [36]
    Hertz J, Krogh A, and Palmer RG (1991) Introduction to theory of neural computation, Addison-Wesley, Reading, MA.Google Scholar
  37. [37]
    Hill T, O’Connor M, Remus W. (1996) Neural network models for time series Models forecasts. Management Sciences 42(7): 1082–1092.zbMATHCrossRefGoogle Scholar
  38. [38]
    Hinton GE (1989) Connectionist learning procedures, Artificial Intelligence, 40: 185–243.CrossRefGoogle Scholar
  39. [39]
    Hopfield JJ (1982) Neural Networks and physical systems with emergent collective computational abilities. Proc. of the Nat. Acad. of Sciences, USA, 79: 2554–2558. Neural Networks Approach 139MathSciNetGoogle Scholar
  40. [40]
    Hopfield JJ (1984) Neurons with graded response have collective computational properties like those of two-state neurons. Proc. of the Nat. Acad. of Sciences, USA 81: 3088–3092.Google Scholar
  41. [41]
    Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are Universal approximators. Neural Networks 2(5): 359–366.CrossRefGoogle Scholar
  42. [42]
    Hu MJC (1964) Application of the ADALINE system to weather forecasting. Master Thesis, Technical Report 6775-1, Stanford El. Lab., Stanford, CA.Google Scholar
  43. [43]
    Ishikawa M. and Moriyama T (1996) Prediction of time series by a structural learning of neural networks.Fuzzy Sets and Systems 82: 167–176.CrossRefGoogle Scholar
  44. [44]
    Iyer MS and Rhinehart RR (2000) A novel methodGoogle Scholar
  45. [45]
    Iyer MS and Rhinehart RR (2000) A Novel Method To Stop Neural Network Training. 2000 American Control Conference, paper WM17-3Google Scholar
  46. [46]
    Johanson EM, Dowla EU, and Goodman DM (1990) Backpropagation learning for multi-layer feedforward neural networks using the conjugate gradient method, Report UCRL-JC-104850, Lawrence Livermore National Laboratory, CA.Google Scholar
  47. [47]
    Jollife IT (1986) Principal Components Analysis. Springer-Verlag.Google Scholar
  48. [48]
    Jordan M (1986) Attractor dynamics and parallelism in a connectionist sequential machine. Proc. of the Eight Annual Conference on Cognitive Science Society:532–546.Google Scholar
  49. [49]
    Karnin ED (1990) A simple procedure for Pruning back-propagation trained neural networks. IEEE Trans on Neural Networks 2: 188–197.Google Scholar
  50. [50]
    Khorasani K and Weng W (1994) Structure Adaptation in Feedforward Neural Networks. Proc. IEEE Internat. Conf. on Neural Networks, III: 1403–1408.Google Scholar
  51. [51]
    Klimasauskas CC (1991) Applying Neural Networks. Part 3: Training a Neural Network. PC-AI, May/June: 20–24. BGoogle Scholar
  52. [52]
    Kohonen T (1989) Self-Organisation and Associative Memory. 3rd Edition, Springer, Berlin, NY.Google Scholar
  53. [53]
    Kröse B and Smagt P (1996) An introduction to neural networks, The University of Amsterdam, Eighth edition, November, Scholar
  54. [54]
    Kubat M (1998) Decision trees can initialise radial-basis-function networks. IEEE Trans. on Neural Networks. 9: 813–821.CrossRefGoogle Scholar
  55. [55]
    Kurita T (1990) A method to determine the number of hidden units of three-layered neural networks by information criteria, Trans. of Inst. of Electronics, Information and Commun. Engineers, J73-D-II-11: 1872–1878 (in Japanese)Google Scholar
  56. [56]
    Lapedes A and Farber R (1988) Nonlinear signal processing using neural networks: Prediction and system modelling. Technical Report LA-UR-87-2662, Los Alamos National Laboratory, Los Alamos, NM.Google Scholar
  57. [57]
    Le Cun Y, Denker JS, and Solla SA (1990) Optimal Brain Damage. In: Touretzky S (Ed.). Advances in Neural Information Processing Systems 2, San Mateo, CA, Morgan Kaufman.Google Scholar
  58. [58]
    Levin AU, Leen TK, and Moody JE (1994) Fast pruning using principle components, In: Advances in Neural Information Processing Systems 6, Covan JD, Tesauro G and Alspector J, Editors: 35–42, Morgan Kaufman Publi. Inc., San Mateo, CA.Google Scholar
  59. [59]
    Lippmann RP (1987) An introduction to computing with neural nets. IEEE ASSP Magazine (April): 4–22Google Scholar
  60. [60]
    Mahmoud E (1984) Accuracy in forecasting: A survey. J. of Forecasting 3:139–159.Google Scholar
  61. [61]
    McClelland JL and Rumelhart DE (1988) Exploration in Parallel Distributed Processing. Cambridge, MA, MIT Press.Google Scholar
  62. [62]
    McCulloch WS, Pitts W, (1943) A logical Calculus of the ideas Immanent in nervous activity. Bulletin of Mathematical Biophysics 5:115–133.MathSciNetzbMATHGoogle Scholar
  63. [63]
    McNees SK (1985) Which forecast should you use. New England Economic Review, July/August: 36–42.Google Scholar
  64. [64]
    Minsky ML and Papert S, (1969) Perceptrons. MIT Press, Cambridge MA.zbMATHGoogle Scholar
  65. [65]
    Moody JE (1991) Note on Generalization, Regularization and Architecture Selection in Nonlinear Systems, Proc. of the IEEE-SP Workshop: 1–10.Google Scholar
  66. [66]
    Moody JE and Darken CJ (1989) Fast learning in networks of locally-tuned processing units. Neural Computation 1: 281–294.Google Scholar
  67. [67]
    Morozov VA (1984) Methods for Solving Incorrectly Posed Problems. Springer-Verlag, Berlin.Google Scholar
  68. [68]
    Mozer MC and Smolensky P (1990) Skeletonization: A technique for trimming the fat from a network via relevance assessment. In: Advances in Neural Information Processing 1, Touretzky DS (Ed.): 107–115.Google Scholar
  69. [69]
    Murata N, Yoshizawa S, and Amari S (1994) Network Information criterion-Determining the number of Hidden Units for an Artificial Neural model. IEEE Trans. On Neural Networks 6: 865–871.Google Scholar
  70. [70]
    Natarajan S and Rhinehart RR (1997) Automated Stopping Criteria For Neural Network Training. Proc. of the 1997 American Control Conf., paper #TP09-4.Google Scholar
  71. [71]
    Nelson M, Hill T, O’Connor M (1994) Can a neural network be applied to time series forecasting and learn seasonal patterns: An empirical investigation. Proc. of the 20th Annual Hawaii Intl. Conf on System Sciences: 649–655.Google Scholar
  72. [72]
    Oja E (1982) A simplified neuron model as a principal component analyzer. Journal of Mathematical Biology 15: 267–273.CrossRefMathSciNetzbMATHGoogle Scholar
  73. [73]
    Palit AK and Popovic D (2000) Nonlinear combination of forecasts using artificial neural network, fuzzy logic and neuro-fuzzy approaches, FUZZ-IEEE, 2: 566–571.Google Scholar
  74. [74]
    Pineda FJ (1987) Generalisation of back-propagation to recurrent neural networks. Physical Review Letters 59: 2229–2232.CrossRefMathSciNetGoogle Scholar
  75. [75]
    Poggio T and Girosi F (1990) Networks for Approximation and Learning. Proc. IEEE 78:1481–1497.CrossRefGoogle Scholar
  76. [76]
    Powel MID (1988) Radial basis function approximation to polynomials, Numerical Analysis Proceedings, Dundee, U.K.: 223–241.Google Scholar
  77. [77]
    Prechelt L (1998) Early Stopping-but when? In: Orr GB and Moeller K-R (Eds.), Neural Networks: Tricks of the Trade. Springer, Berlin: 55–69.Google Scholar
  78. [78]
    Reed R (1993) Pruning Algorithms-A Survey. IEEE Trans. on Neural Networks 4: 740–747.CrossRefGoogle Scholar
  79. [79]
    Rosenblatt F, (1958) The Perceptron: A probabilistic model for information storage and organisation of the brain. Psych. Review 65: 386–408.MathSciNetGoogle Scholar
  80. [80]
    Rumelhart DE and McClelland (1986) Parallel Distributed Processing: Explorations in the Microstructure of Cognition MIT Press, Cambridge, MA.Google Scholar
  81. [81]
    Rumelhart DE, Hinton GE, and Williams RJ (1986) Learning internal representation by back-propagation errors. In: Rumelhart DE, McClelland JL, the PDP Research Group(Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition. MIT Press, MA.Google Scholar
  82. [82]
    Sastry PS, Santharam G, and Unikrishnan KP (1994) Memory neuron networks for identification and control of dynamic systems. IEEE Trans. on NeuralGoogle Scholar
  83. [83]
    Schmidhuber J (1989) Accelerated learning in backpropagation net, In: Connectionism in Perspective, Elsevier, North Holland, Amsterdam, pp. 439–445.Google Scholar
  84. [84]
    Sharda R and Patil RB (1990) Neural Networks as Forecasting Experts: An Empirical Test, Proc. of the IJCNN Meeting, Washington: 491–494.Google Scholar
  85. [85]
    Shi S and Liu B (1993) Nonlinear combination of forecasts with neural networks. Proc. of Intl. Joint Conf. on Neural Networks’ 93 (IJCNN’ 93), Nagoya, Japan, 952–962.Google Scholar
  86. [86]
    Silva FM and Almeida LB (1990) Speeding-up backpropagation, In: Advances of Neural Computers, Eds. Eckmiller R, Elsevier Science Publish. BV., North Holland, pp. 151–158.Google Scholar
  87. [87]
    Specht DF (1988) Probabilistic neural networks for classification, or associative memory, Proc. of IEEE Intern. Conf. on Neural Networks, San Diego, 1: 525–532.Google Scholar
  88. [88]
    Specht DF (1990) Probabilistic neural networks and the polynomial ADALINE as complementary techniques for classifications. IEEE Trans. on Neural Networks, 1: 111–121.CrossRefGoogle Scholar
  89. [89]
    Sprecher DA (1965) On the Structure of Continuous Functions of Several Variables. Trans. Amer. Math. Soc. 115:340–355.MathSciNetzbMATHGoogle Scholar
  90. [90]
    Srinivasan D, Liew AC, Chang CS (1994) A neural network short-term load forecaster. Electric Power Systems Research 28: 227–234.CrossRefGoogle Scholar
  91. [91]
    Stahlberger A and Riedmuller M (1996) Fast network pruning and feature extraction using the Unit-OBS algorithm. Advances in Neural Information Processing systems (NIPS’96), Denver.Google Scholar
  92. [92]
    Stone M (1977) An asymptotic equivalence of choice of model by cross-validation and Akaike’s criterion cross validation. J. of the Royal Statistical Soc. B36:44–47.Google Scholar
  93. [93]
    Sue CT, Tong LI, and Leou CM (1997) Combination of time series and neural network for reliability forecasting modelling. J. Chin. Inst. Ind. Eng. 14(4): 419–429.Google Scholar
  94. [94]
    Tang Z, Almeida de Ch, and Fishwick, PA (1991) Time series forecasting using neural networks vs. Box-Jenkins methodology. Simulation 57(5): 303–310.Google Scholar
  95. [95]
    Tiao GC and Tsay RS (1989) Model specification in multivariate time series. J. of the Royal Statistical Society B 51: 157–213.MathSciNetzbMATHGoogle Scholar
  96. [96]
    Tikhonov AN (1963) On solving incorrectly posed problems and methods of regularisation. Docklady Akademii Nauk USSR 151: 501–504.zbMATHGoogle Scholar
  97. [97]
    Tseng F-M, Yu H-Ch, and Tzeng G-H (2002) Combining neural network model with seasonal time series ARIMA model. Technological Forecasting.Google Scholar
  98. [98]
    Vapnik V (1995) The Nature of Statistical Learning Theory, Springer-Verlag, NY.zbMATHGoogle Scholar
  99. [99]
    Villiers de J and Bernard E (1992) Backpropagation Neural Nets with one and Two Hidden Layers. IEEE Trans. On Neural Networks: 136–141.Google Scholar
  100. [100]
    Vogl TP, Mangis JK, Rigler AK, Zink WT and Allcon DL (1988) Accelerating the convergence of backpropagation method, Biological Cybernetics, vol. 59: 257–263.CrossRefGoogle Scholar
  101. [101]
    Voort VD, Dougherty M, and Watson M. (1996) Combining Kohonen Maps with ARIMA time series models to forecast traffic flow. Transp. Res. Circ. (Emerg. Technol.) 4C(5): 307–318.Google Scholar
  102. [102]
    Wedding II DK and Cios KJ (1996) Time series forecasting by combining RBF networks certainty factors, and the Box-Jenkins model. Neurocomputing 10: 149–168.CrossRefzbMATHGoogle Scholar
  103. [103]
    Weigend AS, Rumelhart DE, and Huberman BA (1991) Generalisation by weightelimination with application to forecasting. Adv. In Neural Information Processing Systems, Morgan Kaufmann, San Mateo, CA 3: 875–882.Google Scholar
  104. [104]
    Werbos P (1990) Backpropagation through time what it does and how to do it, Proc. of IEEE, 78(10):1550–1560.Google Scholar
  105. [105]
    Werbos PJ (1974) Beyond Regression: New Tool for Prediction and analysis in the Behavioural sciences. Ph.D. Thesis, Harvard University, Cambridge, MA.Google Scholar
  106. [106]
    Werbos PJ (1989) Backpropagation and neural control: A review and prospectus. Internat. Joint Conf. of Neural Networks, Washington, 1: 209–216.Google Scholar
  107. [107]
    Widrow B and Hoff ME (1960) Adaptive Switching Circuits. In: Anderson J and Rosenfeld E. (eds.) Neurocomputing. MIT Press, Cambridge, MA, 126–134.Google Scholar
  108. [108]
    Williams RJ and Zipser D (1989) A learning algorithm for continually running fully recurrent neural networks. Neural Computation 1: 270–280.Google Scholar
  109. [109]
    Winkler R and Makridakis S (1983) The combination of forecasts, Journal of the Royal Statistical Society, Series A: 150–157.Google Scholar
  110. [110]
    Yang Y (2000) Combining different procedures for adaptive regression, J. of Multivar. Analysis, 74: 135–161.zbMATHGoogle Scholar
  111. [111]
    Yu X-H, Chen G-A, and Cheng S-X (1995) Dynamic Learning Rate Optimization of the Backpropagation Algorithm. IEEE Trans. on Neural Networks 3: 669–677.Google Scholar
  112. [112]
    Zhang PG (2003) Time series forecasting using a hybrid ARIMA and neural network models. Neurocomputing 50:159–175.zbMATHGoogle Scholar
  113. [113]
    Zhou S, Popovic D, and Schulz-Ekloff G (1991) An Improved Learning Law for Backpropagation Networks. IEEE Int. Conf. on Neural Networks, San Francisco: 573–579.Google Scholar

Selected Reading

  1. [114]
    Anderson JA (1972) A Simple Neural Network Generating an Interactive Memory, Mathematical Biosciences 14: 197–220.CrossRefzbMATHGoogle Scholar
  2. [115]
    Cybenko G (1988) Continuous valued neural networks with two hidden layers are sufficient. Technical Report, Taft University.Google Scholar
  3. [116]
    Kohonen T (1972) Correlation Matrix Memories. IEEE Transactions on Computers 21: 353–359.zbMATHCrossRefGoogle Scholar
  4. [117]
    Kolmogorov AI (1957) On Representation of Continuous Function of Many Variables by Superposition of Continuous Functions of One Variable and Addition. Dokl. Akad. Nauk USSR 114:953–956.MathSciNetzbMATHGoogle Scholar
  5. [118]
    Kurkova V (1991) Kolmogorov’s Theorem is Relevant, Neural Computation, 3: 617–622.CrossRefGoogle Scholar
  6. [119]
    Kurkova V (1992) Kolmogorov’s Theorem and Multilayer Neural Networks, Neural Networks 5: 501–506.CrossRefGoogle Scholar
  7. [120]
    Moody JE (1992) The Effective Number of Parameters: An Analysis of Generalization and Regularisation in Nonlinear learning Systems. In: Advances in Neural Information Processing 4 (Moody JE, Hanson SJ, and Lippmann RP (Eds.), Morgan Kaufman Publ., San Mateo, CA.Google Scholar
  8. [121]
    Schwenkler F, Kestler H, Palm G (2001) Three learning phases for radial-basisfunction networks. Neural Networks 14: 439–458.Google Scholar
  9. [122]
    Schwenkler F, Kestler H, Palm G, and Höher M (1994) Similarities of LVQ and RBF learning, Proc. IEEE International Conference SMC: 646–651.Google Scholar
  10. [123]
    Xiaosong D, Popovic D, and Schulz-Ekloff G (1995) Oscillation-Resisting in the Learning of Backpropagation Neural Networks. 3rd IFAC/IFIP Workshop on Algorithms and Architectures for Real-Time Control, 31 May-2 June, Ostend, Belgium.Google Scholar

Copyright information

© Springer-Verlag London Limited 2005

Personalised recommendations