Abstract
This paper presents a novel technique to separate the pattern representation in each hidden layer to facilitate many classification tasks. This technique requires that all patterns in the same class will have near representions and the patterns in different classes will have distant representions. This requirement is applied to any two data patterns to train a selected hidden layer of the MLP or the RNN. The MLP can be trained layer by layer feedforwardly to accomplish resolved representations. The trained MLP can serve as a kind of kernel functions for categorizing multiple classes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: A Learning Algorithm for Boltzmann Machines. Cognitive Science 9, 147–169 (1985)
Boser, B.E., Guyon, I.M., Vapnik, V.N.: A Training Algorithm for Optimal Margin Classifier. In: Proceedings of the Fifth Annual Workshop on Computational learning theory, pp. 144–152 (1992)
Cleeremans, A., Servan-Schreiber, D., McClelland, J.L.: Finite State Automata and Simple Recurrent Networks. Neural Computation 1, 372–381 (1989)
Elman, J.L.: Finding Structure in Time. Cognitive Science 14, 179–211 (1990)
Hopcroft, J.E., Ullman, J.D.: Introduction to Automata Theory, Languages, and Computation. Addison-Wesley, Reading (1979)
Liou, C.Y., Lin, S.L.: The Other Variant Boltzmann Machine. In: International Joint Conference on Neural Networks, IJCNN, Washington, D.C., USA, June 18-22, pp. 449–454 (1989)
Liou, C.Y., Yu, W.J.: Ambiguous Binary Representation in Multilayer Neural Network. In: Proceedings of ICNN, Perth, Australia, November 27 - December 1, vol. 1, pp. 379–384 (1995)
Liou, C.Y., Yuan, S.K.: Error Tolerant Associative Memory. Biological Cybernetics 81, 331–342 (1999)
Liou, C.Y., Chen, H.T., Huang, J.C.: Separation of Internal Representations of the Hidden Layer. In: Proceedings of the International Computer Symposium, ICS, Workshop on Artificial Intelligence, Chiayi, Taiwan, pp. 26–34 (2000)
Liou, C.Y., Lin, S.L.: Finite Memory Loading in Hairy Neurons. Natural Computing 5, 15–42 (2006)
Liou, C.Y.: Backbone Structure of Hairy Memory. In: Kollias, S., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4131, pp. 688–697. Springer, Heidelberg (2006)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning Internal Representations by Error Propagation. In: Rumelhart, D.E., McClelland, J.L. (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, pp. 318–362. MIT Press, Cambridge (1986)
Tomita, M.: Dynamics Construction of Finite-State Automata from Examples Using Hill-Climing. In: Proceedings of the Fourth Annual Conference of the Cognitive Science Society, pp. 105–108 (1982)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liou, CY., Cheng, WC. (2008). Resolving Hidden Representations. In: Ishikawa, M., Doya, K., Miyamoto, H., Yamakawa, T. (eds) Neural Information Processing. ICONIP 2007. Lecture Notes in Computer Science, vol 4985. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69162-4_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-69162-4_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69159-4
Online ISBN: 978-3-540-69162-4
eBook Packages: Computer ScienceComputer Science (R0)