Abstract
Learning at multiple resolutions provides a fast, hierarchical and efficient technique for extracting models from empirical data. In this chapter we describe the application of wavelets for multi-resolution learning in artificial neural networks and inductive decision trees, and show how wavelets may provide a unifying framework for various supervised learning techniques. A Wave-Net is an artificial neural network with activation functions derived from the class of wavelets. Wave- Nets combine the mathematically rigorous, multi-resolution character of wavelets with the adaptive learning of artificial neural networks. Learning with Wave-Nets is efficient, and is explicitly based on the local or global error of approximation. The advantages of Wave-Net learning over other artificial neural learning techniques are highlighted, and learning methods for minimizing the L2 or L∞ norms are described. The reduced black box character of Wave-Nets is demonstrated by the explicit relationship between Wave-Net parameters and the quality of learning, and by the ability to extract if-then rules from a Haar Wave-Net. The relationship between Haar Wave-Nets and other rule-extraction techniques such as decision trees is described.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Adomaitis, R. A., Farber, R. M., Hudson, J. L., Kevrekidis, I. G., Kube, M., Lapedes, A. S., Application of Neural Nets to System Identification and Bifurcation Analysis of Real World Experimental Data, Los Alamos National Laboratory Technical Report, LA-UR-90–515 (1990)
Bakshi, B. R., Multi-Resolution Methods for Modeling, Analysis and Control of Chemical Process Operations, Ph. D. thesis, Department of Chemical Engineering, Massachusetts Institute of Technology (1992)
Bakshi, B. R., Stephanopoulos, G., Wave-Net: a Multiresolution, Hierarchical Neural Network with Localized Learning, AIChE J., 39, 57–81 (1993)
Barto, A. G., Connectionist Learning for Control, in Neural Networks for Control, MIT Press, Cambridge (1991)
Bhat, N. V., Minderman, P. A., McAvoy, T., Wang, N. S., Modeling Chemical Process Systems via Neural Computation, IEEE Control Systems Magazine, 1 24–30, April (1990)
Cormen, T. H., Leiserson, C. E., Rivest, R. L., Introduction to Algorithms, The MIT Press, Cambridge, MA (1990)
Daubechies, I., Ten Lectures on Wavelets, SIAM (1992)
Daubechies, I., The Wavelet Transform, Time-Frequency Localization and Signal Analysis, IEEE Trans. Inform. Theory, 36, 5, 961–1005 (1990)
Haesloop, D., Holt, B. R., A Neural Network Structure for System Identification, Proc. Amer. Control Conf., 2460–2465 (1990)
Hernandez, E., Arkun, Y., A Study of the Control Relevant Properties of Backpropagation Neural Net Models of Nonlinear Dynamical Systems, Comp. Chem. Eng., 16, 227 (1992)
Hoskins, J. C., Himmelblau, D. M., Artificial Neural network Models of Knowledge Representation in Chemical Engineering, Comp. Chem. Eng.12, 881–890(1988)
Koulouris, A., Bakshi, B. R., Stephanopoulos, G., Learning an unknown function from Examples: Analysis and Extension of the Wave-Net Solution, in preparation (1993)
Kramer, M. A., Leonard, J. A., Diagnosis Using Backpropagation Neural Networks - Analysis and Criticism, Comp. Chem. Eng., 14, 12, 1323–1338(1990)
Leonard, J. A., Kramer, M. A., Radial Basis Function Networks for Classifying Process Faults, IEEE Control Systems, 14,31–38, April (1991)
Ljung, L., System Identification, Prentice Hall (1987)
Mallat, S., A Theory for Multiresolution Signal Decomposition: The Wavelet Representation, IEEE Trans. Pat. Anal. Mach. Intel., 11, 7, 674–693 (1989)
Mardia, K., Kent, J., Bibby, J.,Multivariate Analysis, Academic Press, London (1980)
Moody, J., Fast Learning in Multi-Resolution Hierarchies, Research Report, Yale University, YALEU/DCS/RR-681 (1989)
Narendra, K., Parthasarthy, K., Identification and Control of Dynamical Systems using Neural Networks, IEEE Trans. Neural Networks,1, 1 (1989)
Poggio, T., Girosi, F., A Theory of Networks for Approximation and Learning, AI Lab., Memo no. 1140, MIT AI Lab, July (1990)
Quinlan, J. R., Induction of Decision Trees, Machine Learning, 1, 1, 81–106 (1986)
Sanger, T. D., A Tree-Structured Algorithm for Reducing Computation in Networks with Separable Basis Functions, Neural Computation, 3, 67–78 (1991)
Saraiva, P., Stephanopoulos, G., Continuous Process Improvement Through Inductive and Analogical Learning, AIChE J., 38, 2,161–183 (1992)
Ungar, L. H., Powell, B. A., Kamens, S. N., Adaptive Networks for Fault Diagnosis and Process Control, Comp. Chem. Eng., 14, 561–573 (1990)
Venkatasubramaniam, V., McAvoy, T. J., Computers and Chemical Engineering, special issue on Neural Networks in Chemical Engineering, 16, 4 (1992)
Venkatasubramaniam, V., Vaidyanathan, R., Yamamoto, Y., Process Fault Detection and Diagnosis Using Neural Networks -I. Steady-State Processes, Comp. Chem. Eng. 14, 7, 699–712 (1990)
Watanabe, K., Matsuura, I., Abe, M., Kubota, M., Himmelblau, D. M., Incipient Fault Diagnosis of Chemical Processes via Artificial Neural Networks, AIChE J., 35,1803–1812 (1989)
Whiteley, J. R., Davis, J. F., Knowledge-Based Interpretation of Sensor Patterns, Comp. Chem. Eng., 16, 4, 329–347 (1992)
Widrow, B., Hoff, M., Adaptive Switching Circuits, IRE WESCON Cony. Record, Part 4, 96–104 (1960)
Ydstie, B. E., Forecasting and Control using Adaptive Connectionist Networks, Comp. Chem. Eng., 14, 4/5, 583–599 (1990)
Zhang, Q., Benveniste, A., Wavelet Networks, IEEE Trans. Neural Networks, 3, 6, 889–898 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1994 Springer Science+Business Media New York
About this chapter
Cite this chapter
Bakshi, B.R., Koulouris, A., Stephanopoulos, G. (1994). Learning at Multiple Resolutions: Wavelets as Basis Functions in Artificial Neural Networks, and Inductive Decision Trees. In: Motard, R.L., Joseph, B. (eds) Wavelet Applications in Chemical Engineering. The Kluwer International Series in Engineering and Computer Science, vol 272. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-2708-4_5
Download citation
DOI: https://doi.org/10.1007/978-1-4615-2708-4_5
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-7923-9461-7
Online ISBN: 978-1-4615-2708-4
eBook Packages: Springer Book Archive