Abstract
A variety of information measures have been introduced for finite alphabet random variables, vectors, and processes:entropy, mutual information, relative entropy, conditional entropy, and conditional mutual information. All of these can be expressed in terms of divergence and hence the generalization of these definitions to infinite alphabets will follow from a general definition of divergence. Many of the properties of generalized information measures will then follow from those of generalized divergence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1990 Springer Science+Business Media New York
About this chapter
Cite this chapter
Gray, R.M. (1990). Relative Entropy. In: Entropy and Information Theory. Springer, New York, NY. https://doi.org/10.1007/978-1-4757-3982-4_5
Download citation
DOI: https://doi.org/10.1007/978-1-4757-3982-4_5
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4757-3984-8
Online ISBN: 978-1-4757-3982-4
eBook Packages: Springer Book Archive