Abstract
Rather than presenting a specific trick, this paper aims at providing a methodology for large scale, real-world classification tasks involving thousands of classes and millions of training patterns. Such problems arise in speech recognition, handwriting recognition and speaker or writer identification, just to name a few. Given the typically very large number of classes to be distinguished, many approaches focus on parametric methods to independently estimate class conditional likelihoods. In contrast, we demonstrate how the principles of modularity and hierarchy can be applied to directly estimate posterior class probabilities in a connectionist framework. Apart from ofiering better discrimination capability, we argue that a hierarchical classification scheme is crucial in tackling the above mentioned problems. Furthermore, we discuss training issues that have to be addressed when an almost infinite amount of training data is available.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
E. B. Baum, D. Haussler (1989) What Size Net Gives Valid Generalization?, Neural Computation 1, pp 151–160.
C. M. Bishop (1995) Training with Noise is Equivalent to Tikhonov Regularization, Neural Computation 7, issue 1, Jan 1995, pp 108–116.
H. Bourlard, N. Morgan (1994) Connectionist Speech Recognition-A Hybrid Approach, Kluwer Academic Press, 1994.
H. Bourlard, N. Morgan (1992) A Context Dependent Neural Network for Continuous Speech Recognition, IEEE Proc. Intl. Conf. on Acoustics, Speech and Signal Processing, volume 2, pp 349–352, San Francisco, CA.
L. Breiman, J. H. Friedman, R. A. Olshen & C. J. Stone (1984) Classification and Regression Trees, Wadsworth International Group, Belmont, CA.
J. Bridle (1990) Probabilistic Interpretation of Feed Forward Classification Network Outputs, with Relationships to Statistical Pattern Recognition, In Neurocomputing: Algorithms, Architectures, and Applications, F. Fogelman-Soulie and J. Hérault, eds. Springer Verlag, New York.
R. Duda, P. Hart (1973) Pattern Classification and Scene Analysis, John Wiley & Sons, Inc.
M. Finke, J. Fritsch, P. Geutner, K. Ries & T. Zeppenfeld (1997) The JanusRTk Switchboard/Callhome 1997 Evaluation System, Proceedings of LVCSR Hub5-e Workshop, May 13–15, Baltimore, Maryland.
H. Franco, M. Cohen, N. Morgan, D. Rumelhart & V. Abrash (1994) Context-Dependent Connectionist Probability Estimation in a Hybrid Hidden Markov Model-Neural Net Speech Recognition System, Computer Speech and Language, Vol. 8, No 3, pp 211–222, July 1994.
J. Fritsch, M. Finke (1997) ACID/HNN: Clustering Hierarchies of Neural Networks for Context-Dependent Connectionist Acoustic Modeling, In Proceedings of International Conference on Acoustics, Speech and Signal Processing, May 1998, Seattle, Wa.
J. Fritsch (1997) ACID/HNN: A Framework for Hierarchical Connectionist Acoustic Modeling, In Proceedings of IEEEWorkshop on Automatic Speech Recognition and Understanding, December 1997, Santa Barbara, Ca.
J. Fritsch, M. Finke & A. Waibel (1997) Context-Dependent Hybrid HME/HMM Speech Recognition using Polyphone Clustering Decision Trees, Intl. Conf. on Acoustics, Speech and Signal Processing, volume 3, pp 1759, Munich, Germany.
J. Fritsch (1996) Modular Neural Networks for Speech Recognition, Tech. Report CMU-CS-96-203, Carnegie Mellon University, Pittsburgh, PA.
M. M. Hochberg, G. D. Cook, S. J. Renals, A. J. Robinson, & R. S. Schechtman (1995) The 1994 ABBOT Hybrid Connectionist-HMM Large-Vocabulary Recognition System, In Spoken Language Systems Technology Workshop, pp 170–176, ARPA, Jan. 1995.
D. J. Kershaw, M. M. Hochberg & A. J. Robinson (1995) Context-Dependent Classes in a Hybrid Recurrent Network-HMM Speech Recognition System, Tech. Rep. CUED/F-INFENG/TR217, Cambridge University Engineering Department, Cambridge, England.
C. J. Merz, P. M. Murphy (1996) UCI Repository of Machine Learning Databases, http://www.ics.uci.edu/mlearn/MLRepository.html, University of California, Department of Information and Computer Science.
N. Morgan, H. Bourlard (1992) Factoring Networks by a Statistical Method, Neural Computation 4, No. 6, pp 835–838, 1992.
N. Morgan, H. Bourlard (1995) An Introduction to Hybrid HMM/Connectionist Continuous Speech Recognition, Signal Processing Magazine, pp 25–42, May 1995.
NIST (1997) Conversational Speech Recognition Workshop, DARPA Hub-5E Evaluation, May 13–15/1997, Baltimore, Maryland.
L. Prechelt (1994) Proben1-A Set of Neural Network Benchmark Problems and Benchmarking Rules, Technical Report 21/94, University of Karlsruhe, Germany.
J. R. Quinlan (1986) Induction of Decision Trees, Machine Learn. 1, pp 81–106.
L. R. Rabiner (1989) A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition, Proceedings of the IEEE 77, pp 257–285.
S. R. Safavian, D. Landgrebe (1991) A Survey of Decision Tree Classifier Methodology, IEEE Transactions on Systems, Man and Cybernetics, Vol. 21, No. 3, pp 660–674.
J. Schürmann, W. Doster (1984) A Decision Theoretic Approach to Hierarchical Classifier Design, Pattern Recognition 17(3), pp 359–369.
J. Schürmann (1996) Pattern Classification: A Unified View of Statistical and Neural Approaches, John Wiley & Sons, Inc., New York, 1996.
J. T. Tou, R. C. Ganzales (1974) Pattern Recognition Principles, Addison Wesley, Reading, Massachusettes.
S. Young (1996) Large Vocabulary Continuous Speech Recognition: a Review, CUED Technical Report, Cambridge University.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Fritsch, J., Finke, M. (1998). Applying Divide and Conquer to Large Scale Pattern Recognition Tasks. In: Orr, G.B., Müller, KR. (eds) Neural Networks: Tricks of the Trade. Lecture Notes in Computer Science, vol 1524. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49430-8_16
Download citation
DOI: https://doi.org/10.1007/3-540-49430-8_16
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65311-0
Online ISBN: 978-3-540-49430-0
eBook Packages: Springer Book Archive