Abstract
This work describes AppART, an ART—based low parameterized neural model that incrementally approximates continuous—valued multidimensional functions from noisy data using biologically plausible processes. AppART performs a higher—order Nadaraya—Watson regression and can be interpreted as a fuzzy system. Some benchmark problems are solved in order to study AppART from an application point of view and to compare its results with the ones obtained from other models.
The authors wish to thank the Dipartimento di Matematica e Informatica of the Università degli Studi di Udine for its support on the conception and elaboration of this work.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Asfour, Y. R., Carpenter, G. A., Grossberg, S., & Lesher, G. W. (1993). Fusion artmap: A neural network architecture for multi-channel data fusion and classification. In Proceedings of WCNN-93, world congress on neural networks (Vol. 2, pp. 210–215 ). Hillsdale, NJ, USA: Lawrence Erlbaum Associates.
Bellman, R. (1961). Adaptive control processes: A guided tour. Princeton: Princeton University Press.
Bianchini, M., Fasconi, P., & Gori, M. (1995). Learning without local minima in radial basis fucntion networks. IEEE Transactions on Neural Networks, 6, 749–756.
Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford: Clarendon Press.
Blake, C. L., & Merz, C. J. (1998). UCI repository of machine learning databases.
Blanzieri, E., Katenkamp, P., & Giordana, A. (1995). Growing radial basis function networks. In Proceedings of the Fourth Workshop on Learing Robots. Karlsruhe, Germany.
Broomhead, D. S., & Lowe, D. (1998). Multivariable functional interpolation and adaptive networks. Complex Systems, 2, 321–329.
Browne, A., & Sun, R. (2002). Connectionist inference models. To appear in Neural Networks.
Cano, J. M., Dimitriadis, Y. A., Aranzo, M. J., & Lopez, J. (1996). FasArt: A new neuro—fuzzy architecture for incremental learning in systems identification. In Proceedings of the 13th World Congress of IFAC (Vol. F, pp. 133–138). San Francisco.
Cano, J. M., Dimitriadis, Y. A., Aranzo, M. J., & Lopez, J. (1997). FasBack: Matching error based learning for automatic generation of fuzzy logic systems. In Proceedings of the sixth IEEE International Conference on Fuzzy Systems (Vol. 3, pp. 1561–1566). Barcelona.
Cano, J. M., Dimitriadis, Y. A., Gomez, E., & LOpez, J. (2001). Learning from noisy information in FasART and FasBack neuro-fuzzy systems. Neural Networks, 14, 407–425.
Carpenter, G. A., & Grossberg, S. (1987a). ART2: stable self-organization of pattern recognition codes for analog inputs patterns. Applied Optics, 26, 4919–4930.
Carpenter, G. A., & Grossberg, S. (1987b). A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics and Image Processing, 37, 54–115.
Carpenter, G. A., & Grossberg, S. (1990). ART3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures. Neural Networks, 3, 129–152.
Carpenter, G. A., & Grossberg, S. (1992). A self-organizing neural network for supervised learning, recognition and prediction. IEEE Comunications Magazine, 30, 38–49.
Carpenter, G. A., & Grossberg, S. (1994). Artificial Intelligence and Neural Networks: Steps Toward Principled Integratio. In V. Honavar and L. Uhr (Eds.), Artificial Intelligence and Neural Networks: Steps Toward Principled Integration. San Diego, CA: Academic Press.
Carpenter, G. A., Grossberg, S., Markuzon, N., Reynolds, J. H., & Rosen, D. (1992). Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Transactions on Neural Networks, 3, 698–713.
Carpenter, G. A., Grossberg, S., & Reynolds, J. H. (1991). ARTMAP: Supervised real-time learning and classification of non-stationary data by a self-organazing neural network. Neural Networks, 4, 565–588.
Carpenter, G. A., Grossberg, S., & Rosen, D. B. (1991). Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Networks, 4, 759–771.
Carpenter, G. A., & Markuzon, N. (1998). ARTMAP-IC and medical diagnosis: Instance counting and inconsistent cases. Neural Networks, 11(2), 323–336.
Carpenter, G. A., Milenova, B. L., & Noeske, B. W. (1998). Distributed ARTMAP: A neural network for fast distributed supervised learning. Neural Networks, 11(5), 793–813.
Carpenter, G. A., & Ross, W. D. (1995). ART-EMAP: A neural network architecture for object recognition by evidence accumulation. IEEE Transactions on Neural Networks, 6 (4), 805–818.
Carpenter, G. A., & Tan, A.-H. (1995). Rule extraction: From neural architecture to symbolic representation. Connection Science, 7 (1), 3–27.
Caudill, M. (1993). GRNN and bear it. AI Expert, 8, 28–33.
Duda, R. O., & Hart, P. E. (1973). Pattern classification and scene analysis. New York: John Wiley.
Esposito, A., Marinaro, M., Oricchio, D., & Scarpetta, S. (2000). Approximation of continuous and discontinuous mappings by a growing neural rbf-based algorithm. Neural Networks, 13, 651–665.
Feldman, J. A., Lakoff, G., Bailey, D. R., Narayanan, S., Regier, T., & Stolcke, A. (1996). Lo — the first five years of an automated language acquisition project. AI Review, 8.
Freeman, J. A., & Skapura, D. M. (1991). Neural networks: algorithms, applications and programming techniques. Reading: Addison-Wesley.
Fritzke, B. (1994a). Fast learning with incremental RBF networks. Neural Processing Letters, 1, 2–5.
Fritzke, B. (1994b). Growing cell structures-A self-organizing network for unsupervised and supervised learning. Neural Networks, 7, 1441–1460.
Geffner, H. (1992). Default reasoning: causal and conditional theories. Cambridge, MA: MIT Press.
Girosi, F., & Poggio, T. (1990). Networks and the best approximation property. Biological Cybernetics, 63, 169–176.
Glover, F., & Laguna, M. (1993). Tabu search. In C. Reeves (Ed.), Modern heurisic techniques for combinatorial problems (pp. 70–150 ). Oxford: Blackwell.
Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning. Reading: Addison-Wesley.
Grossberg, S. (1980). How does the brain build a cognitive code? Psycologial Review, 87, 1–51.
Grossberg, S. (1982). Studies of Mind and Brain: Neural Principles of Learning, Perception, Development, Cognition, and Motor Control. Boston: Reidel.
Grossberg, S., & Williamson, J. R. (1996). A self-organizing system for classifying complex images: Natural texture and synthetic aperture radar (CAS/CNS-TR-96–002). Boston, MA: Boston University.
Honavar, V. (1994). Symbolic artificial intelligence and numeric artificial neural networks: Toward a resolution of the dichotomy. In R. Sun and L. Bookman (Eds.), Computational Architectures Integrating Symbolic and Neural Processes (pp. 351–388 ). New York: Kluwer.
Hoya, T., & Constantinides, A. G. (1998). An heuristic pattern correction scheme for GRNNs and its application to speech recognition. In Proceedings of the IEEE Workshop on Neural Networks for Signal Processing (pp. 351–359 ). Cambridge, U.K.
Hwang, Y. S., & Bang, S. Y. (1997). An efficient method to construct a radial basis function neural network classifier. Neural Networks, 8, 1495–1503.
Kakas, A., Kowalski, R., & Toni, F. (1993). Abductive logic programming. Journal of Logic and Computation, 6(2), 719–770.
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671–680.
Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics,. 43, 59–69.
Kosko, B. (1997). Fuzzy engineering. New York: Prentice Hall.
Lawrence, S., Tsoi, A. C., & Black, A. D. (1996). Function approximation with neural networks and local methods: Bias, variance and smoothness. In P. Bartlett, A. Burkitt, and R. Williamson (Eds.), Australian Conference on Neural Networks, ACNN’96 (pp. 16–21). Australian National University.
Mackey, M. C., & Glass, L. (1977). Oscillation and chaos in physiological control systems. Science, 197–287.
Manna, Z., & Waldinger, R. (1985). The Logical Basis for Computer Programming (Vol. 1: Deductive Reasoning). Reading: Addison-Wesley.
Marriott, S., & Harrison, R. F. (1995). A modified fuzzy ARTMAP architecture for the approximation of noisy mappings. Neural Networks, 8, 619–641.
Martinetz, T. M., Berkovich, S. G., & Shulten, K. J. (1993). Neural-Gas network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4, 558–560.
McCulloch, W. S., & Pitts, W. (1943). A logical calculus of ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5, 115–133.
McGarry, K., Wermter, S., & Maclntyre, J. (1999). Hybrid neural systems: from single coupling to fully integrated neural networks. Neural Computing Surveys, 2, 62–93.
Millân, J. R. (1994). Learining efficient reactive behavioral sequences from basic reflexes in a goal-oriented autonomous robot. In From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior (pp. 266–274 ). Cambridge, MA: MIT Press.
Moody, J., & Darken, C. (1989). Fast learning in networks of locally-tuned processing units. Neural Computation, 1, 281–294.
Musavi, M., Ahmed, W., Chan, K., Faris, K., & Hummels, D. (1992). On the training of radial basis function classifiers. Neural Networks, 5, 595–603.
Nadaraya, E. A. (1964). On estimating regression. Theory of Probability and Its Application, 10, 186–190.
O’Neill, M. C. (1989). Escherichia coli promoters: I. Consensus as it relates to spacing class, specificity, repeat substructure and three dimensional organization. Journal of Biologial Chemistry, 264, 5522–5530.
Orr, M. J. L. (1995). Regularization in the seleccion of radial basis function centers. Neural Computation, 7, 606–620.
Park, J., & Sandberg, I. W. (1991). Universal approximation using radial basis function. Neural Computation, 3, 246–257.
Poggio, T., & Girosi, F. (1990). Networks for approximation and learning. Proceedings of the IEEE, 78, 1481–1496.
Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1, 81–106.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Boston: MIT Press.
Schioler, H., & Hartman, U. (1992). Mapping neural network derived from the Parzen window estimator. Neural Networks, 5, 903–909.
Smolensky, P. (1988). On the proper treatment of connectionism. Behavioral and Brain Sciences, 11, 1-74.
Spetch, D. (1990). A general regression neural network. IEEE Transactions on Neural Networks, 2, 568–578.
Srinivasa, N. (1997). Learning and generalization of noisy mappings using a modified PROBART neural network. IEEE Transactions on Signal Processing, 45(10), 2533–2550.
Sun, R. (1994). Integrating Rules and Connectionism for Robust Commonsense Reasoning. New York: Wiley.
Tan, A.-H. (1997). Cascade ARTMAP: Integration neural computation and symbolic knowledge processing. IEEE Transactions on Neural Networks, 8(2), 237–250.
Tan, A.-H. (2000). Innovation in ART Neural Networks. In J. Jain (Ed.), Innovation in ART Neural Networks. CRC Press.
Tomandl, D., & Schober, A. (2001). A Modified General Regression Neural Network (MGRNN) with new, efficient training algorithms as a robust `back box’—tool for data analysis. Neural Networks, 14, 1023–1034.
Towell, G., & Shavlik, J. (1990). Directed propagation of training signals through knowledge—based neural networks (Tech. Rep. No. CS-TR-1990989). Madison, WI: University of Wisconsin, Computer Sciences Department.
Towell, G., & Shavlik, J. (1993). Extracting rules from knowledge-based neural networks. Machine Learning, 13(1), 71–101.
Towell, G. G., & Shavlik, J. W. (1994). Knowledge-based artificial neural networks,. Artificial Intelligence, 70, 119–165.
Verzi, S. J., Heileman, G. L., Georgiopoulos, M., & Healy, M. J. (1998).Boosted ARTMAP. In Proceedings of the International Joint Conference on Neural Networks (IJCNN-98) (Vol. 1, pp. 396–401). Alaska.
Wermter, S., & Sun, R. (2000). Hybrid Neural Systems. Heidelberg: Springer.
Williamson, J. R. (1996). Gaussian artmap: A neural network for fast incremental learning of noisy multidimensional maps. Neural Networks, 9, 881–897.
Williamson, J. R. (1997). A constructive, incremental-learning network for mixture modeling and clasiffication. Neural Computation, 9, 1517–1543.
Williamson, J. R. (1998). A neural model for self-organizing feature detectors and classifiers in a network hierarchy (CAS/CNS TR-98–033). Boston, MA: Boston University.
Wilson, D. R., & Martinez, T. R. (1997). Improved heterogeneous distance functions. Journal of Artificial Intelligence Research, 6, 1–34.
Wilson, R., & Martinez, T. R. (1996).Heterogeneous radial basis function networks. In Proceedings of the International Conference on Neural Networks (ICNN’96) (Vol. 2, pp. 1263–1267). Washington DC.
Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8, 338–353.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Martí, L., Policriti, A., García, L. (2002). AppART: An ART Hybrid Stable Learning Neural Network for Universal Function Approximation. In: Abraham, A., Köppen, M. (eds) Hybrid Information Systems. Advances in Soft Computing, vol 14. Physica, Heidelberg. https://doi.org/10.1007/978-3-7908-1782-9_9
Download citation
DOI: https://doi.org/10.1007/978-3-7908-1782-9_9
Publisher Name: Physica, Heidelberg
Print ISBN: 978-3-7908-1480-4
Online ISBN: 978-3-7908-1782-9
eBook Packages: Springer Book Archive