Probability and Random Processes

  • Robert M. Gray


In this chapter we develop basic mathematical models of discrete time random processes. Such processes are also called discrete time stochastic processes, information sources, and time series. Physically a random process is something that produces a succession of symbols called “outputs” in a random or nondeterministic manner. The symbols produced may be real numbers such as produced by voltage measurements from a transducer, binary numbers as in computer data, two-dimensional intensity fields as in a sequence of images, continuous or discontinuous waveforms, and so on. The space containing all of the possible output symbols is called the alphabet of the random process, and a random process is essentially an assignment of a probability measure to events consisting of sets of sequences of symbols from the alphabet. It is useful, however, to treat the notion of time explicitly as a transformation of sequences produced by the random process. Thus in addition to the common random process model we shall also consider modeling random processes by dynamical systems as considered in ergodic theory.


Probability Measure Random Process Probability Space Measurable Space Event Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    R. B. Ash, Real Analysis and Probability, Academic Press, New York, 1972.Google Scholar
  2. 2.
    P. Billingsley, Ergodic Theory and Information, Wiley, New York, 1965.zbMATHGoogle Scholar
  3. 3.
    L. Breiman, Probability, Addison-Wesley, Menlo Park, Calif., 1968.zbMATHGoogle Scholar
  4. 4.
    J. C. Candy, “A use of limit cycle oscillations to obtain robust analog-to-digital converters,” IEEE Transactions on Communications, vol. COM-22, pp. 298–305, March 1974.CrossRefGoogle Scholar
  5. 5.
    K. L. Chung, A Course in Probability Theory, Academic Press, New York, 1974.zbMATHGoogle Scholar
  6. 6.
    H. Cramer, Mathematical Methods of Statistics, Princeton University Press, Princeton, NJ, 1946.zbMATHGoogle Scholar
  7. 7.
    M. Denker, C. Grillenberger, and K. Sigmund, Ergodic Theory on Compact Spaces, 57, Lecture Notes in Mathematics, Springer-Verlag, New York, 1970.Google Scholar
  8. 8.
    N. A. Friedman, Introduction to Ergodic Theory, Van Nostrand Reinhold Company, New York, 1970.zbMATHGoogle Scholar
  9. 9.
    R. M. Gray, “Oversampled Sigma-Delta Modulation,” IEEE Transactions on Communications, vol. COM-35, pp. 481–489, April 1987.zbMATHCrossRefGoogle Scholar
  10. 10.
    P. R. Halmos, Measure Theory, Van Nostrand Reinhold, New York, 1950.zbMATHGoogle Scholar
  11. 11.
    P. R. Halmos, Lectures on Ergodic Theory, Chelsea, New York, 1956.zbMATHGoogle Scholar
  12. 12.
    H. Inose and Y. Yasuda, “A unity bit coding method by negative feedback,” Proceedings of the IEEE, vol. 51, pp. 1524–1535, November 1963.CrossRefGoogle Scholar
  13. 13.
    A. N. Kolmogorov, Foundations of the Theory of Probability, Chelsea, New York, 1950.Google Scholar
  14. 14.
    U. Krengel, Ergodic Theorems, De Gruyter Series in Mathematics, De Gruyter, New York, 1985.Google Scholar
  15. 15.
    M. Loève, Probability Theory, D. Van Nostrand, Princeton, New Jersey, 1963. Third Edition.zbMATHGoogle Scholar
  16. 16.
    L. E. Maistrov, Probability Theory: A Historical Sketch, Academic Press, New York, 1974. Translated by S. Kotz.zbMATHGoogle Scholar
  17. 17.
    D. Ornstein, Ergodic Theory, Randomness, and Dynamical Systems, Yale University Press, New Haven, 1975.Google Scholar
  18. 18.
    K. Petersen, Ergodic Theory, Cambridge University Press, Cambridge, 1983.zbMATHCrossRefGoogle Scholar
  19. 19.
    P. C. Shields, The Theory of Bernoulli Shifts, The University of Chicago Press, Chicago, Ill., 1973.zbMATHGoogle Scholar
  20. 20.
    Ya. G. Sinai, Introduction to Ergodic Theory, Mathematical Notes, Princeton University Press, Princeton, 1976.zbMATHGoogle Scholar
  21. 21.
    P. Walters, Ergodic Theory-Introductory Lectures, Lecture Notes in Mathematics No. 458, Springer-Verlag, New York, 1975.zbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1988

Authors and Affiliations

  • Robert M. Gray
    • 1
  1. 1.Department of Electrical EngineeringStanford UniversityStanfordUSA

Personalised recommendations