Abstract
Various statistically-based classification methods use entropy functions for error bounds evaluation and feature selection. A lot of such entropy functions are known, each of them providing a measure of “distance” between probability distributions. But in practice, these probability distributions are never precisely known, they are estimated by means of the training set.
In this paper, we introduce one concept of entropy measure e, which is not related to probabilities; e is directly expressed in terms of frequencies, in such a way that e is sensitive to the sample size.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arimoto, S. Information Theoretical Considerations and Estimation Problems. Information and Control 19, 181, 1971.
Chen, C.H. On the Use of Distance and Information Measures in Pattern Recognition and Applications. Pattern Recognition Theory and Application, NATO Advanced Study Inst. Series, K.S. Fu and A.B. Whinston ed. 45, 1977.
Daroczy, Z. Generalised Information Functions. Information and Control 16, 36, 1970.
Devijver, P. On Information Measure in Identification and Parameter Estimation. Proc. 3rd IFAC Symp. Identification and System Parameter Estimation, 631, 1973.
Devijver, P. Entropies of Degree 3 and Lower Bounds for the Average Error Rate. Information and Control 34, 222, 1977.
Gaillat, G. Une Procedure Statistique de Decision avec Apprentissage et son Application a la Reconnaissance de Caracteres Manuscrits. These 3eme cycle, Paris V I, 1975.
Terrenoire, M. Tounissoux, D. Consideration of the Data Uncertainty with Regard to the Sequential Decision Scheme. Pattern Recognition Theory and Applications NATO Advanced Study Inst. Series., K.S. Fu and A.B. Whinston ed, 91, 1977.
Terrenoire M, Tounissoux D. Une Technique de Reconnaissance de Forme pour des Ensembles d’Apprentissage de Petite Taille. Seminaire IRIA. Classification automatique et perception par ordinateur, 113, 1979.
Tounissoux, D. Processus Sequentiel s Adaptatifs de Reconnaissance de Formes pour l’Aide au Diagnostic. These, Lyon, 1980.
Vajda, I. Bounds of the Minimal Error Probability on Checking a Finite or Countable Number of Hypotheses. Problemy Peredachi Informatsii, Vol. 4, No. 1, 9, 1968.
Vajda I. Limit Theorems for Total Variation of Cartesian Product Measures. Studia Scientiarum Mathematicarum Hungarica, 6, 317, 1971.
Van Tree. Decision, Estimation and Modulation Theory. Wiley, 1968.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1982 D. Reidel Publishing Company
About this paper
Cite this paper
Terrenoire, M., Tounissoux, D. (1982). Sample Size Sensitive Entropy. In: Kittler, J., Fu, K.S., Pau, LF. (eds) Pattern Recognition Theory and Applications. NATO Advanced Study Institutes Series, vol 81. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-7772-3_5
Download citation
DOI: https://doi.org/10.1007/978-94-009-7772-3_5
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-009-7774-7
Online ISBN: 978-94-009-7772-3
eBook Packages: Springer Book Archive