Abstract
In the present chapter, we set out the methods for computation of entropy of many random variables or of a stochastic process in discrete and continuous time.
From a fundamental and practical points of view, of particular interest are the stationary stochastic processes and their information-theoretic characteristics, specifically their entropy. Such processes are relatively simple objects, particularly a discrete process, i.e. a stationary process with discrete states and running in discrete time. Therefore, this process is a very good example for demonstrating the basic points of the theory, and so we shall start from its presentation.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Doob, J.: Stochastic Processes. Wiley Publications in Statistics. Wiley, New York (1953)
Doob, J.: Stochastic Processes (Translation to Russian). Inostrannaya Literatura, Moscow (1956)
Ryzhik, J.M., Gradstein, I.S.: Tables of Series, Products and Integrals. Gostekhizdat, Moscow (1951, in Russian)
Ryzhik, J.M., Gradstein, I.S.: Tables of Series, Products and Integrals (Translation from Russian). Academic, New York (1965)
Stratonovich, R.L.: Conditional Markov Processes and Their Application to the Theory of Optimal Control. Moscow State University, Moscow (1966, in Russian)
Stratonovich, R.L.: Amount of information and entropy of segments of stationary Gaussian processes. Problemy Peredachi Informacii 3(2), 3–21 (1967, in Russian)
Stratonovich, R.L.: Conditional Markov Processes and Their Application to the Theory of Optimal Control (Translation from Russian). Modern Analytic and Computational Methods in Science and Mathematics. Elsevier, New York (1968)
Stratonovich, R.L., Grishanin, B.A.: Game problems with constraints of an informational type. Izv. USSR Acad. Sci. Tech. Cybern. 1, 3–12 (1968, in Russian)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Belavkin, R.V., Pardalos, P.M., Principe, J.C., Stratonovich, R.L. (2020). Computation of entropy for special cases. Entropy of stochastic processes. In: Belavkin, R., Pardalos, P., Principe, J. (eds) Theory of Information and its Value. Springer, Cham. https://doi.org/10.1007/978-3-030-22833-0_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-22833-0_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22832-3
Online ISBN: 978-3-030-22833-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)