Advertisement

AppART: An ART Hybrid Stable Learning Neural Network for Universal Function Approximation

  • Luis Martí
  • Alberto Policriti
  • Luciano García
Part of the Advances in Soft Computing book series (AINSC, volume 14)

Abstract

This work describes AppART, an ART—based low parameterized neural model that incrementally approximates continuous—valued multidimensional functions from noisy data using biologically plausible processes. AppART performs a higher—order Nadaraya—Watson regression and can be interpreted as a fuzzy system. Some benchmark problems are solved in order to study AppART from an application point of view and to compare its results with the ones obtained from other models.

Keywords

Radial Basis Function Network General Regression Neural Network Adaptive Resonance Theory Fuzzy ARTMAP Function Approximation Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Asfour, Y. R., Carpenter, G. A., Grossberg, S., & Lesher, G. W. (1993). Fusion artmap: A neural network architecture for multi-channel data fusion and classification. In Proceedings of WCNN-93, world congress on neural networks (Vol. 2, pp. 210–215 ). Hillsdale, NJ, USA: Lawrence Erlbaum Associates.Google Scholar
  2. Bellman, R. (1961). Adaptive control processes: A guided tour. Princeton: Princeton University Press.MATHGoogle Scholar
  3. Bianchini, M., Fasconi, P., & Gori, M. (1995). Learning without local minima in radial basis fucntion networks. IEEE Transactions on Neural Networks, 6, 749–756.Google Scholar
  4. Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford: Clarendon Press.Google Scholar
  5. Blake, C. L., & Merz, C. J. (1998). UCI repository of machine learning databases. Google Scholar
  6. Blanzieri, E., Katenkamp, P., & Giordana, A. (1995). Growing radial basis function networks. In Proceedings of the Fourth Workshop on Learing Robots. Karlsruhe, Germany.Google Scholar
  7. Broomhead, D. S., & Lowe, D. (1998). Multivariable functional interpolation and adaptive networks. Complex Systems, 2, 321–329.MathSciNetGoogle Scholar
  8. Browne, A., & Sun, R. (2002). Connectionist inference models. To appear in Neural Networks.Google Scholar
  9. Cano, J. M., Dimitriadis, Y. A., Aranzo, M. J., & Lopez, J. (1996). FasArt: A new neuro—fuzzy architecture for incremental learning in systems identification. In Proceedings of the 13th World Congress of IFAC (Vol. F, pp. 133–138). San Francisco.Google Scholar
  10. Cano, J. M., Dimitriadis, Y. A., Aranzo, M. J., & Lopez, J. (1997). FasBack: Matching error based learning for automatic generation of fuzzy logic systems. In Proceedings of the sixth IEEE International Conference on Fuzzy Systems (Vol. 3, pp. 1561–1566). Barcelona.Google Scholar
  11. Cano, J. M., Dimitriadis, Y. A., Gomez, E., & LOpez, J. (2001). Learning from noisy information in FasART and FasBack neuro-fuzzy systems. Neural Networks, 14, 407–425.Google Scholar
  12. Carpenter, G. A., & Grossberg, S. (1987a). ART2: stable self-organization of pattern recognition codes for analog inputs patterns. Applied Optics, 26, 4919–4930.Google Scholar
  13. Carpenter, G. A., & Grossberg, S. (1987b). A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics and Image Processing, 37, 54–115.Google Scholar
  14. Carpenter, G. A., & Grossberg, S. (1990). ART3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures. Neural Networks, 3, 129–152.CrossRefGoogle Scholar
  15. Carpenter, G. A., & Grossberg, S. (1992). A self-organizing neural network for supervised learning, recognition and prediction. IEEE Comunications Magazine, 30, 38–49.CrossRefGoogle Scholar
  16. Carpenter, G. A., & Grossberg, S. (1994). Artificial Intelligence and Neural Networks: Steps Toward Principled Integratio. In V. Honavar and L. Uhr (Eds.), Artificial Intelligence and Neural Networks: Steps Toward Principled Integration. San Diego, CA: Academic Press.Google Scholar
  17. Carpenter, G. A., Grossberg, S., Markuzon, N., Reynolds, J. H., & Rosen, D. (1992). Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Transactions on Neural Networks, 3, 698–713.CrossRefGoogle Scholar
  18. Carpenter, G. A., Grossberg, S., & Reynolds, J. H. (1991). ARTMAP: Supervised real-time learning and classification of non-stationary data by a self-organazing neural network. Neural Networks, 4, 565–588.CrossRefGoogle Scholar
  19. Carpenter, G. A., Grossberg, S., & Rosen, D. B. (1991). Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Networks, 4, 759–771.CrossRefGoogle Scholar
  20. Carpenter, G. A., & Markuzon, N. (1998). ARTMAP-IC and medical diagnosis: Instance counting and inconsistent cases. Neural Networks, 11(2), 323–336.Google Scholar
  21. Carpenter, G. A., Milenova, B. L., & Noeske, B. W. (1998). Distributed ARTMAP: A neural network for fast distributed supervised learning. Neural Networks, 11(5), 793–813.Google Scholar
  22. Carpenter, G. A., & Ross, W. D. (1995). ART-EMAP: A neural network architecture for object recognition by evidence accumulation. IEEE Transactions on Neural Networks, 6 (4), 805–818.CrossRefGoogle Scholar
  23. Carpenter, G. A., & Tan, A.-H. (1995). Rule extraction: From neural architecture to symbolic representation. Connection Science, 7 (1), 3–27.CrossRefGoogle Scholar
  24. Caudill, M. (1993). GRNN and bear it. AI Expert, 8, 28–33.Google Scholar
  25. Duda, R. O., & Hart, P. E. (1973). Pattern classification and scene analysis. New York: John Wiley.MATHGoogle Scholar
  26. Esposito, A., Marinaro, M., Oricchio, D., & Scarpetta, S. (2000). Approximation of continuous and discontinuous mappings by a growing neural rbf-based algorithm. Neural Networks, 13, 651–665.CrossRefGoogle Scholar
  27. Feldman, J. A., Lakoff, G., Bailey, D. R., Narayanan, S., Regier, T., & Stolcke, A. (1996). Lo — the first five years of an automated language acquisition project. AI Review, 8.Google Scholar
  28. Freeman, J. A., & Skapura, D. M. (1991). Neural networks: algorithms, applications and programming techniques. Reading: Addison-Wesley.Google Scholar
  29. Fritzke, B. (1994a). Fast learning with incremental RBF networks. Neural Processing Letters, 1, 2–5.CrossRefGoogle Scholar
  30. Fritzke, B. (1994b). Growing cell structures-A self-organizing network for unsupervised and supervised learning. Neural Networks, 7, 1441–1460.CrossRefGoogle Scholar
  31. Geffner, H. (1992). Default reasoning: causal and conditional theories. Cambridge, MA: MIT Press.Google Scholar
  32. Girosi, F., & Poggio, T. (1990). Networks and the best approximation property. Biological Cybernetics, 63, 169–176.Google Scholar
  33. Glover, F., & Laguna, M. (1993). Tabu search. In C. Reeves (Ed.), Modern heurisic techniques for combinatorial problems (pp. 70–150 ). Oxford: Blackwell.Google Scholar
  34. Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning. Reading: Addison-Wesley.MATHGoogle Scholar
  35. Grossberg, S. (1980). How does the brain build a cognitive code? Psycologial Review, 87, 1–51.Google Scholar
  36. Grossberg, S. (1982). Studies of Mind and Brain: Neural Principles of Learning, Perception, Development, Cognition, and Motor Control. Boston: Reidel.Google Scholar
  37. Grossberg, S., & Williamson, J. R. (1996). A self-organizing system for classifying complex images: Natural texture and synthetic aperture radar (CAS/CNS-TR-96–002). Boston, MA: Boston University.Google Scholar
  38. Honavar, V. (1994). Symbolic artificial intelligence and numeric artificial neural networks: Toward a resolution of the dichotomy. In R. Sun and L. Bookman (Eds.), Computational Architectures Integrating Symbolic and Neural Processes (pp. 351–388 ). New York: Kluwer.Google Scholar
  39. Hoya, T., & Constantinides, A. G. (1998). An heuristic pattern correction scheme for GRNNs and its application to speech recognition. In Proceedings of the IEEE Workshop on Neural Networks for Signal Processing (pp. 351–359 ). Cambridge, U.K.Google Scholar
  40. Hwang, Y. S., & Bang, S. Y. (1997). An efficient method to construct a radial basis function neural network classifier. Neural Networks, 8, 1495–1503.Google Scholar
  41. Kakas, A., Kowalski, R., & Toni, F. (1993). Abductive logic programming. Journal of Logic and Computation, 6(2), 719–770.Google Scholar
  42. Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671–680.Google Scholar
  43. Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics,. 43, 59–69.Google Scholar
  44. Kosko, B. (1997). Fuzzy engineering. New York: Prentice Hall.Google Scholar
  45. Lawrence, S., Tsoi, A. C., & Black, A. D. (1996). Function approximation with neural networks and local methods: Bias, variance and smoothness. In P. Bartlett, A. Burkitt, and R. Williamson (Eds.), Australian Conference on Neural Networks, ACNN’96 (pp. 16–21). Australian National University.Google Scholar
  46. Mackey, M. C., & Glass, L. (1977). Oscillation and chaos in physiological control systems. Science, 197–287.Google Scholar
  47. Manna, Z., & Waldinger, R. (1985). The Logical Basis for Computer Programming (Vol. 1: Deductive Reasoning). Reading: Addison-Wesley.MATHGoogle Scholar
  48. Marriott, S., & Harrison, R. F. (1995). A modified fuzzy ARTMAP architecture for the approximation of noisy mappings. Neural Networks, 8, 619–641.Google Scholar
  49. Martinetz, T. M., Berkovich, S. G., & Shulten, K. J. (1993). Neural-Gas network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4, 558–560.Google Scholar
  50. McCulloch, W. S., & Pitts, W. (1943). A logical calculus of ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5, 115–133.MathSciNetMATHCrossRefGoogle Scholar
  51. McGarry, K., Wermter, S., & Maclntyre, J. (1999). Hybrid neural systems: from single coupling to fully integrated neural networks. Neural Computing Surveys, 2, 62–93.Google Scholar
  52. Millân, J. R. (1994). Learining efficient reactive behavioral sequences from basic reflexes in a goal-oriented autonomous robot. In From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior (pp. 266–274 ). Cambridge, MA: MIT Press.Google Scholar
  53. Moody, J., & Darken, C. (1989). Fast learning in networks of locally-tuned processing units. Neural Computation, 1, 281–294.CrossRefGoogle Scholar
  54. Musavi, M., Ahmed, W., Chan, K., Faris, K., & Hummels, D. (1992). On the training of radial basis function classifiers. Neural Networks, 5, 595–603.Google Scholar
  55. Nadaraya, E. A. (1964). On estimating regression. Theory of Probability and Its Application, 10, 186–190.CrossRefGoogle Scholar
  56. O’Neill, M. C. (1989). Escherichia coli promoters: I. Consensus as it relates to spacing class, specificity, repeat substructure and three dimensional organization. Journal of Biologial Chemistry, 264, 5522–5530.Google Scholar
  57. Orr, M. J. L. (1995). Regularization in the seleccion of radial basis function centers. Neural Computation, 7, 606–620.CrossRefGoogle Scholar
  58. Park, J., & Sandberg, I. W. (1991). Universal approximation using radial basis function. Neural Computation, 3, 246–257.CrossRefGoogle Scholar
  59. Poggio, T., & Girosi, F. (1990). Networks for approximation and learning. Proceedings of the IEEE, 78, 1481–1496.CrossRefGoogle Scholar
  60. Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1, 81–106.Google Scholar
  61. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Boston: MIT Press.Google Scholar
  62. Schioler, H., & Hartman, U. (1992). Mapping neural network derived from the Parzen window estimator. Neural Networks, 5, 903–909.CrossRefGoogle Scholar
  63. Smolensky, P. (1988). On the proper treatment of connectionism. Behavioral and Brain Sciences, 11, 1-74.Google Scholar
  64. Spetch, D. (1990). A general regression neural network. IEEE Transactions on Neural Networks, 2, 568–578.Google Scholar
  65. Srinivasa, N. (1997). Learning and generalization of noisy mappings using a modified PROBART neural network. IEEE Transactions on Signal Processing, 45(10), 2533–2550.Google Scholar
  66. Sun, R. (1994). Integrating Rules and Connectionism for Robust Commonsense Reasoning. New York: Wiley.MATHGoogle Scholar
  67. Tan, A.-H. (1997). Cascade ARTMAP: Integration neural computation and symbolic knowledge processing. IEEE Transactions on Neural Networks, 8(2), 237–250. Google Scholar
  68. Tan, A.-H. (2000). Innovation in ART Neural Networks. In J. Jain (Ed.), Innovation in ART Neural Networks. CRC Press.Google Scholar
  69. Tomandl, D., & Schober, A. (2001). A Modified General Regression Neural Network (MGRNN) with new, efficient training algorithms as a robust `back box’—tool for data analysis. Neural Networks, 14, 1023–1034. Google Scholar
  70. Towell, G., & Shavlik, J. (1990). Directed propagation of training signals through knowledge—based neural networks (Tech. Rep. No. CS-TR-1990989). Madison, WI: University of Wisconsin, Computer Sciences Department.Google Scholar
  71. Towell, G., & Shavlik, J. (1993). Extracting rules from knowledge-based neural networks. Machine Learning, 13(1), 71–101. Google Scholar
  72. Towell, G. G., & Shavlik, J. W. (1994). Knowledge-based artificial neural networks,. Artificial Intelligence, 70, 119–165.Google Scholar
  73. Verzi, S. J., Heileman, G. L., Georgiopoulos, M., & Healy, M. J. (1998).Boosted ARTMAP. In Proceedings of the International Joint Conference on Neural Networks (IJCNN-98) (Vol. 1, pp. 396–401). Alaska.Google Scholar
  74. Wermter, S., & Sun, R. (2000). Hybrid Neural Systems. Heidelberg: Springer.CrossRefGoogle Scholar
  75. Williamson, J. R. (1996). Gaussian artmap: A neural network for fast incremental learning of noisy multidimensional maps. Neural Networks, 9, 881–897.Google Scholar
  76. Williamson, J. R. (1997). A constructive, incremental-learning network for mixture modeling and clasiffication. Neural Computation, 9, 1517–1543.CrossRefGoogle Scholar
  77. Williamson, J. R. (1998). A neural model for self-organizing feature detectors and classifiers in a network hierarchy (CAS/CNS TR-98–033). Boston, MA: Boston University.Google Scholar
  78. Wilson, D. R., & Martinez, T. R. (1997). Improved heterogeneous distance functions. Journal of Artificial Intelligence Research, 6, 1–34.Google Scholar
  79. Wilson, R., & Martinez, T. R. (1996).Heterogeneous radial basis function networks. In Proceedings of the International Conference on Neural Networks (ICNN’96) (Vol. 2, pp. 1263–1267). Washington DC.Google Scholar
  80. Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8, 338–353. Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Luis Martí
    • 1
    • 2
  • Alberto Policriti
    • 1
  • Luciano García
    • 2
  1. 1.Dipartimento di Matematica e InformaticaUniversità degli Studi di UdineUdineItaly
  2. 2.Facultad de Matemática y ComputaciónUniversidad de La HabanaLa HabanaCuba

Personalised recommendations