Communication Systems pp 3788  Cite as
Probability Theory and Random Processes
 404 Downloads
Abstract
The theory of sets, in its more general form, began with Georg Cantor (1845–1918) in the nineteenth century. Cantor established the basis for this theory and demonstrated some of its most important results, including the concept of set cardinality. Cantor was born in St. Petersburg, Russia, but lived most of his life in Germany (Boyer 1974). The ideas relative to the notions of universal set, empty set, set partition, discrete systems, continuous systems and infinity are, in reality, as old as philosophy itself.
2.1 Set Theory, Functions, and Measure
The theory of sets, in its more general form, began with Georg Cantor (1845–1918) in the nineteenth century. Cantor established the basis for this theory and demonstrated some of its most important results, including the concept of set cardinality. Cantor was born in St. Petersburg, Russia, but lived most of his life in Germany (Boyer 1974). The ideas relative to the notions of universal set, empty set, set partition, discrete systems, continuous systems and infinity are, in reality, as old as philosophy itself.
In the time of Zenon, one of the most famous preSocratic philosophers, the notion of infinity was already discussed. Zenon, considered the creator of the dialectic, was born in Elea, Italy, around 504 B.C. and was the defendant of Parmenides, his master, against criticism from the followers of Pythagoras. Pythagoras was born in Samos around 580 B.C. and created a philosophic current based on the quantification of the universe.
For the Pythagoreans, unity itself is the result of a being and of a not being. It can be noticed that the concept of emptiness is expressed in the above sentence as well as the concept of a universal set. The Pythagoreans established an association between the number one and the point, between the number two and the line, between the number three and the surface, and between the number four and the volume (de Souza 1996). In spite of dominating the notion of emptiness, the Greeks still did not have the concept of zero.
Zenon, by his turn, defended the idea of a unique being continuous and indivisible, of Parmenides, against the multiple being, discontinuous and divisible of Pythagoras. Aristotle presents various of Zenon’s arguments relative to movement, with the objective of establishing the concept of a continuum.
Aristotle was born in Estagira, Macedonia, in the year 384 B.C. The first of Aristotle’s arguments suggests the idea of an infinite set, to be discussed later (Durant 1996). This argument is as follows: If a mobile object has to cover a given distance, in a continuous space, it must then cover first the first half of the distance before covering the whole distance.
Despite the reflections of preSocratic philosophers and of others that followed, no one had yet managed to characterize infinite until 1872. In that year, J. W. R. Dedekind (1831–1916) pointed to the universal property of infinite sets, which has found applications as far as in the study of fractals (Boyer 1974):In effect, length and time and in general all contents are called infinite in two senses, whether meaning division or whether with respect to extremes. No doubt, infinities in quantity can not be touched in a finite time; but infinities in division, yes, since time itself is also infinite in this manner. As a consequence, it is in an infinite time and not in a finite time that one can cross infinite and, if infinities are touched, they are touched by infinities and not by finite.
Cantor also recognized the fundamental property of sets but, differing from Dedekind, he noticed that not all infinite sets are equal. This notion originated the cardinal numbers, which will be covered later, in order to establish a hierarchy of infinite sets in accordance with their respective powers. The results of Cantor led him to establish set theory as a fully developed subject. As a consequence of his results on transfinite arithmetic, too advanced for his time, Cantor suffered attacks of mathematicians like Leopold Kronecker (1823–1891), who disregarded him for a position at the University of Berlin.A system S is called infinite when it is similar to a part of itself. On the contrary, S is said to be finite.
Set TheoryThe new transfinite arithmetic is the most extraordinary product of human thought, and one of the most beautiful achievements of human activity in the domain of the purely intelligible.
The notion of a set is of fundamental importance and is axiomatic—in a manner that a set does not admit a problemfree definition, i.e., a definition which will not resort to the original notion of a set. The mathematical concept of a set can be used as fundamental for all known mathematics.
Set theory will be developed based on a set of axioms, called fundamental axioms: Axiom of Extension, Axiom of Specification, Peano’s Axioms, Axiom of Choice, besides Zorn’s Lemma, and Schröder–Bernstein’s Theorem (Halmos 1960).
The objective of this section is to develop the theory of sets in an informal manner, just quoting these fundamental axioms, since this theory is used as a basis for establishing a probability measure. Some examples of common sets are given next.

The set of faces of a coin: \(A = \{ C_H, C_T \}\);

The binary set: \(B = \{ 0, 1 \}\);

The set of natural numbers: \(N = \{ 1, 2, 3, \dots \}\);

The set of integer numbers: \(Z = \{ \dots , 1, 2, 0, 1, 2, 3, \dots \}\);
The most important relations in set theory are the belonging relation, denoted as \(a \in A\), in which a is an element of the set A, and the inclusion relation, \(A \subset B\), which is read “A is a subset of the set B,” or B is a superset of the set A.
Sets may be specified by means of propositions as, for example, “The set of students that return their homework on time,” or more formally \(A = \{ a\,  \, a\) return their homework on time \(\}\). This is, in a few cases, a way of denoting the empty set! Alias, the empty set can be written formally as \(\emptyset = \{ a\,  \, a \ne a \}\), i.e., the set the elements of which are not equal to themselves.
The notion of a universal set is of fundamental interest. A universal set is understood as that set which contains all other sets of interest. An example of a universal set is provided by the sample space in probability theory, usually denoted as S or \(\Omega \). The empty set is that set which contains no element and which is usually denoted as \(\emptyset \) or \(\{\ \}\). It is implicit that the empty set is contained in any set, i.e., that \(\emptyset \subset A\), for any given set A. However, the empty set is not in general an element of any other set.

The operation \(\overline{A} \) represents the complement of A with respect to the sample space \(\Omega \);

The subtraction of sets, denoted \(C = A  B\), gives as a result the set the elements of which belong to A and do not belong to B.
Note: If B is completely contained in A : \(A  B\) = \(A \cap \overline{B}\);

The set of elements belonging to A and to B, but not belonging to \((A\,\cap \,B)\) é is specified by \(A\,\Delta \,B = A\,\cup \,B  A\,\cap \,B\).
The generalization of these concepts to families of sets, as, for example, \(\cup _{i=1}^{N}A_{i}\) and \(\cap _{i=1}^{N}A_{i}\), is immediate. The following properties are usually employed as axioms in developing the theory of sets (Lipschutz 1968).

Idempotent
\(A\,\cup \,A\,=\,A , \qquad A\,\cap \,A\,=\,A\).

Associative
\((A\,\cup \,B)\,\cup \,C\,=\,A\,\cup \,(B\,\cup \,C) , \qquad (A\,\cap \,B)\,\cap \,C\,=\,A\,\cap \,(B\,\cap \,C)\).

Commutative
\(A\,\cup \,B\,=\,B\,\cup \,A , \qquad A\,\cap \,B\,=\,B\,\cap \,A\).

Distributive
\(A\,\cup \,(B\,\cap \,C)\,=\,(A\,\cup \,B)\,\cap \,(A\,\cup \,C)\) ,
\(A\,\cap \,(B\,\cup \,C)\,=\,(A\,\cap \,B)\,\cup \,(A\,\cap \,C)\).

Identity
\(A\,\cup \,\emptyset \,=\,A , \qquad A\,\cap \,U\,=\,A\)
\(A\,\cup \,U\,=\,U , \qquad A\,\cap \,\emptyset \,=\,\emptyset \).

Complementary
\(A\,\cup \, \overline{A}\,=\,U , \qquad A\,\cap \, \overline{A}\,=\,\emptyset \) \(\overline{( \overline{A})}\,=\,A\)
\(U\,=\,\emptyset , \qquad \overline{\emptyset }\,=\,U\).

de Morgan laws
\( \overline{A\,\cup \,B}\,=\, \overline{A}\,\cap \, \overline{B} , \qquad \overline{A\,\cap \,B}\,=\, \overline{A}\,\cup \, \overline{B}\).
Families of Sets
Indexing
The Cartesian product is a way of expressing the idea of indexing of sets. The indexing of sets expands the possibilities for the use of sets, allowing to produce eventually entities known as vectors and signals.
Application 1: Consider \(B_i = \{0,1\}\). Starting from this set, it is possible to construct an indexed sequence of sets by defining its indexing: \(\{B_{i \epsilon I} \} \), \( I=\{0,\ldots ,7\}\). This family of indexed sets \(B_{i}\) constitutes a finite discrete sequence, i.e., a vector. The ASCII set of binary symbols is an example.
Application 2: Consider \(B_i = \{A,A\}\), but now let \(I=Z \) (the set of positive and negative integers plus zero). It follows that \(\{B_{i \epsilon Z} \}\), which represents an infinite series of –A’s and A’s, i.e., it represents a binary digital signal. For example, Fig. 2.5 represents a discrete signal in both amplitude and time.
Application 3: Still letting \(B_i = \{A,A\}\), but considering the indexing over the set of real numbers, \(\{B_{i \epsilon I} \}\), in which \(I = R\), a signal is formed which is discrete in amplitude but it is continuous in time, such as the telegraph signal in Fig. 2.6.
Algebra of Sets
In order to construct an algebra of sets or, equivalently, to construct a field over which operations involving sets make sense, a few properties have to be obeyed.
 (1)
If \(A\,\in \,\mathcal {F}\) then \(\overline{A}\,\in \,\mathcal {F}\). A is the set containing desired results, or over which one wants to operate;
 (2)
If \(A\,\in \,\mathcal {F}\) and \( B\,\in \,\mathcal {F}\) then \( A\,\cup \,B\ \ \in \, \mathcal {F}\).
The above properties guarantee the closure of the algebra with respect to finite operations over sets. It is noticed that the universal set \(\Omega \) always belongs to the algebra, i.e., \(\Omega \in \mathcal {F}\), because \(\Omega = A \cup \overline{A}\). The empty set also belongs to the algebra, i.e., \(\emptyset \in \mathcal {F}\), since \( \emptyset = \overline{\Omega }\), followed by property 1.
Example: The family \(\{\emptyset , \,\Omega \}\) complies with the above properties and therefore represents an algebra. In this case \(\emptyset = \{\}\) and \(\overline{\emptyset } = \Omega \). The union is also represented, as can be easily checked.
Example: Given the sets \(\{C_{H}\}\) and \(\{C_{T}\}\), representing the faces of a coin, respectively, if \(\{C_H\} \in \mathcal {F}\) then \( \{\overline{C_{H}}\} = \{C_{T}\}\in \mathcal {F}\). It follows that \( \{C_{H},C_{T}\} \in \mathcal {F}\) \(\Rightarrow \Omega \in \mathcal {F}\) \(\Rightarrow \emptyset \in \mathcal {F}\).
The previous example can be translated by the following expression. If there is a measure for heads, then there must be also a measure for tails, in order for the algebra to be properly defined. Whenever a probability is assigned to an event, then a probability must also be assigned to the complementary event.
The cardinality of a finite set is defined as the number of elements belonging to this set. Sets with an infinite number of elements are said to have the same cardinality if they are equivalent, i.e., \(A \sim B\) if \(\sharp A = \sharp B\). Some examples of sets and their respective cardinals are presented next.

\(I = \{1,\ldots ,k\} \Rightarrow C_{I}=k\);

\(N = \{0,1,\ldots \} \Rightarrow C_{N}\) or \(\aleph _0\);

\(Z = \{\ldots ,2,1,0,1,2,\ldots \} \Rightarrow C_{Z}\);

\(Q = \{\ldots ,1/3,0,1/3,1/2,\ldots \} \Rightarrow C_{Q}\);

\(R = (\infty ,\infty ) \Rightarrow C_{R}\) or \(\aleph \).
For the above examples, the following relations are verified: \(C_{R}>C_{Q}=C_{Z}=C_{N}>C_{I}\). The notation \(\aleph _0\), for the cardinality of the set of natural numbers, was employed by Cantor.
The cardinality of the power set, i.e., of the family of sets consisting of all subsets of a given set I, \(\mathcal {F}\) = \(2^{I}\), is \(2^{C_I}\).
Borel Algebra
The Borel algebra \(\mathcal {B}\), or \(\sigma \)algebra, is an extension of the algebra so far discussed to operate with limits at infinity. The following properties are required from a \(\sigma \)algebra.
 1
\(A\,\in \,\mathcal {B}\) \(\Rightarrow \overline{A}\,\in \,\mathcal {B}\),
 2
\(A_{i}\,\in \,\mathcal {B}\) \(\Rightarrow \bigcup _{i=1}^{\infty }A_{i}\,\in \,\mathcal {B}\).
The above properties guarantee the closure of the \(\sigma \)algebra with respect to enumerable operations over sets. These properties allow the definition of limits in the Borel field.
2.2 Probability Theory
This section summarizes the more basic definitions related to probability theory, random variables and stochastic processes, the main results, and conclusions of which will be used in subsequent chapters.
Probability theory began in France with studies about games of chance. Antoine Gombaud (1607–1684), known as Chevalier de Méré, was very keen on card games and would discuss with Blaise Pascal (1623–1662) about the probabilities of success in this game. Pascal, also interested in the subject, began a correspondence with Pierre de Fermat (1601–1665) in 1654, which originated the theory of finite probability (Zumpano and de Lima 2004).
However, the first known work about probability is De Ludo Aleae (About Games of Chance), by the Italian medical doctor and mathematician Girolamo Cardano (1501–1576), published in 1663, almost 90 years after his death. This book was a handbook for players, containing some discussion about probability.
The first published treatise about the Theory of Probability, dated 1657, was written by the Dutch scientist Christian Huygens (1629–1695), a folder titled De Ratiociniis in Ludo Aleae (About Reasoning in Games of Chance).
Another Italian, the physicist and astronomer Galileo Galilei (1564–1642), was also concerned with random events. In a fragment probably written between 1613 and 1623, entitled Sopra le Scorpete dei Dadi (About Dice Games), Galileo answers a question asked, it is believed, by the Grand Duke of Tuscany: When three dice are thrown, although both the number 9 and the number 10 may result from six distinct manners, in practice, the chances of getting a 9 are lower than those of obtaining a 10. How can that be explained?
The six distinct manners by which these numbers (9 and 10) can be obtained are (1 3 6), (1 4 5), (2 3 5), (2 4 4), (2 6 2) and (3 3 4) for the number 10 and (1 2 6), (1 3 5), (1 4 4), (2 2 5), (2 3 4) and (3 3 3) for the number 9. Galileo concluded that, for this game, the permutations of the triplets must also be considered since (1 3 6) and (3 1 6) are distinct possibilities. He then calculated that in order to obtain the number 9 there are in fact 25 possibilities, while there are 27 possibilities for the number 10. Therefore, combinations leading to the number 10 are more frequent.
Abraham de Moivre (1667–1754) was another important mathematician for the development of probability theory. He wrote a book of great influence at the time, called Doctrine of Chances. The law of large numbers was discussed by Jacques Bernoulli (1654–1705), Swiss mathematician, in his work Ars Conjectandi (The Art of Conjecturing).
The study of probability was deepened in the eighteenth and nineteenth centuries, being worth of mentioning the works of French mathematicians PierreSimon de Laplace (1749–1827) and Siméon Poisson (1781–1840), as well as the German mathematician Karl Friedrich Gauss (1777–1855).
Axiomatic Approach to Probability
Probability theory is usually presented in one of the following manners: The classical approach, the relative frequency approach, and the axiomatic approach. The classical approach is based on the symmetry of an experiment, but employs the concept of probability in a cyclic manner, because it is defined only for equiprobable events. The relative frequency approach to probability is more recent and relies on experiments.
Considering the difficulties found in the two previous approaches to probability, respectively, the cyclic definition in the first case and the problem of convergence in a series of experiments for the second case, henceforth only the axiomatic approach will be followed in this text. Those readers interested in the classical or in the relative frequency approach are referred to the literature (Papoulis 1981).
 Axiom 1

\(P(S) = 1\), in which S denotes the sample space or universal set and \(P(\cdot )\) denotes the associated probability.
 Axiom 2

\(P(A) \ge 0\), in which A denotes an event belonging to the sample space.
 Axiom 3

\(P(A \cup B) = P(A) + P(B)\), in which A and B are mutually exclusive events and \(A \cup B\) denotes the union of events A and B.
Using his axiomatic approach to probability theory, Kolmogorov established a firm mathematical basis on which other theories rely as, for example, the Theory of Stochastic Processes, Communications Theory, and Information Theory.
Kolmogorov’s fundamental work was published in 1933, in Russian, and soon afterward was published in German with the title Grundbegriffe der Wahrscheinlichkeits Rechnung (Fundamentals of Probability Theory) (James 1981). In this work, Kolmogorov managed to combine Advanced Set Theory, of Cantor, with Measure Theory, of Lebesgue, in order to produce what to this date is the modern approach to probability theory.

If A is independent of B, then \(P(A  B) = P(A)\). It then follows that \(P(BA) = P(B)\) and that B is independent of A.

If \(B \subset A\), then \(P(A  B) = 1\).

If \(A \subset B\), then \(P(A  B) = \frac{P(A)}{P(B)} \ge P(A)\).

If A and B are independent events then \(P(A\,\cap \,B) = P(A)\cdot P(B)\).

If \(P(A) = 0\) or \(P(A) = 1\), then event A is independent of itself.

If \(P(B) = 0\), then P(AB) can assume any arbitrary value. Usually in this case one assumes \(P(A  B) = P(A)\).

If events A and B are disjoint, and nonempty, then they are dependent.
2.3 Random Variables
A random variable (r.v.) X represents a mapping of the sample space on the line (the set of real numbers). A random variable is usually characterized by a cumulative probability function (CPF) \(P_X(x)\), or by a probability density function (pdf) \(p_X(x)\).
2.3.1 Average Value of a Random Variable
2.3.2 Moments of a Random Variable

\(m_{1} = E[X]\), arithmetic mean, average value, average voltage, statistical mean;

\(m_{2} = E[X^{2}]\), quadratic mean, total power;

\(m_{3} = E[X^{3}]\), measure of asymmetry of the probability density function;

\(m_{4} = E[X^{4}]\), measure of flatness of the probability density function.
2.3.3 The Variance of a Random Variable
2.3.4 The Characteristic Function of a Random Variable
2.3.4.1 Some Important Random Variables
 (1)
Gaussian random variable
The random variable X with pdfis called a Gaussian (or Normal) random variable. The Gaussian random variable plays an extremely important role in engineering, considering that many wellknown processes can be described or approximated by this pdf. The noise present in either analog or digital communications systems usually can be considered Gaussian as a consequence of the influence of many factors (LeonGarcia 1989). In (2.13), \(m_{X}\) represents the average value and \(\sigma ^{2}_{X}\) represents the variance of X. Figure 2.9 illustrates the Gaussian pdf and its corresponding cumulative probability function.$$\begin{aligned} p_{X}(x) = \frac{1}{ \sigma _{X} \sqrt{2 \pi } } e^{  \frac{ ( x  m_X )^2 }{ 2 \sigma _X^2 } } \end{aligned}$$(2.13)  (2)
Rayleigh random variable
An often used model to represent the behavior of the amplitudes of signals subjected to fading employs the following pdf (Kennedy 1969), (Proakis 1990):known as the Rayleigh pdf, with average \(E[X] = \sigma \sqrt{ \pi /2}\) and variance \(V[X] = (2  \pi ) \frac{ \sigma ^{2} }{2}\).$$\begin{aligned} p_{X}( x ) = \frac{ x }{ \sigma ^{2}} e^{  \frac{ x^{2}}{2 \sigma ^{2}}} u( x ) \end{aligned}$$(2.14)The Rayleigh pdf represents the effect of multiple signals, reflected or refracted, which are captured by a receiver, in a situation in which there is no main signal component or main direction of propagation (Lecours et al. 1988). In this situation, the phase distribution of the received signal can be considered uniform in the interval \((0,2\pi )\). It is noticed that it is possible to closely approximate a Rayleigh pdf by considering only six waveforms with independently distributed phases (Schwartz et al. 1966).
 (3)
Sinusoidal random variable
A sinusoidal tone X has the following pdf:The pdf and the CPF of X are illustrated in Fig. 2.10.$$\begin{aligned} p_{X}(x) = \frac{1}{ \pi \sqrt{V^{2}  x^{2}}} , \ x < V. \end{aligned}$$(2.15)
Joint Random Variables
The r.v.’s X and Y are called uncorrelated if \( E[XY] = E[X] E[Y]\). The criterion of statistical independence of random variables, which is stronger than that for the r.v.’s being uncorrelated, is satisfied if \(p_{XY}(x,y) = p_{X}(x).p_{Y}(y)\).
2.4 Stochastic Processes
A random process (or stochastic process) X(t) defines a random variable for each point on the time axis. A stochastic process is said to be stationary if the probability densities associated with the process are timeindependent.
The Autocorrelation Function
However, since the signal is timevarying, its statistical mean can also change with time, as illustrated in Fig. 2.16. In the example considered, the variance is initially diminishing with time and later it is growing with time. In this case, an adjustment in the signal variance, by means of an automatic gain control mechanism, can remove the pdf dependency on time.
 (1)
Stationary mean \(\Rightarrow \ \ m_{X}(t)\,=\,m_{X}\);
 (2)
Stationary power \(\Rightarrow \ \ P_{X}(t)\,=\,P_{X}\) ;
 (3)
Firstorder stationarity implies that the firstorder moment is also timeindependent;
 (4)
Secondorder stationarity implies that the secondorder moments are also timeindependent;
 (5)
Narrowsense stationarity implies that the signal is stationary for all orders, i.e., \(p_{X_{1}\cdots X_{M}}(x_{1},\ldots ,x_{M};t)\,=\,p_{X_{1}\ldots X_{M}}(x_{1},\ldots ,x_{M})\)
WideSense Stationarity
The following conditions are necessary to guarantee that a stochastic process is widesense stationary.
 (1)
The autocorrelation is timeindependent;
 (2)
The mean and the power are constant;
 (3)
\(R_X(t_1,t_2) = R_{X}(t_{2}\,\,t_{1})\,=\,R_{X}(\tau )\). The autocorrelation depends on the time interval and not on the origin of the time interval.
Stationarity Degrees
Ergodic Signals

Ergodicity of the mean: \(\overline{X(t)}\,\sim \,E[X(t)]\);

Ergodicity of the power: \(\overline{X^{2}(t)}\,\sim \,\overline{R_{X}}(\tau )\,\sim \,R_{X}(\tau )\);

Ergodicity of the autocorrelation: \(\overline{R_{X}}(\tau )\,\sim \,R_{X}(\tau )\).
A strictly stationary stochastic process has timeindependent joint pdf’s of all orders. A stationary process of second order is that process for which all means are constant and the autocorrelation depends only on the measurement time interval.
Summarizing, a stochastic process is ergodic whenever its statistical means, which are functions of time, can be approximated by their corresponding time averages, which are random processes, with a standard deviation which is close to zero. The ergodicity may appear only on the mean value of the process, in which case the process is said to be ergodic on the mean.
Properties of the Autocorrelation
 (1)
\(R_{X}(0)\,=\,E[X^{2}(t)]\,=\,P_{X}\), (Total power);
 (2)
\(R_{X}(\infty )\,=\,\lim _{\tau \rightarrow \infty }R_{X}(\tau )\) = \(\lim _{\tau \rightarrow \infty }E[X(t\,+\,\tau )X(t)]\,=\,E^2[X(t)]\), (Average power or DC level);
 (3)
Autocovariance: \(C_{X}(\tau )\,=\,R_{X}(\tau )\,\,E^2[X(t)]\);
 (4)
Variance: V[\(X(t)]\,=\,E[(X(t)\,\,E[X(t)])^2]\,=\,E[X^2(t)]\,\,E^2[X(t)]\)
or \(P_{AC}(0)\,=\,R_{X}(0)\,\,R_{X}(\infty )\);
 (5)
\(R_{X}(0)\,\ge \,  R_{X}(\tau )  \), (Maximum at the origin);
This property is demonstrated by considering the following tautology:Thus,$$ E[(X(t)\,\,X(t\,+\,\tau ))^2]\,\ge \,0 . $$i.e.,$$ E[X^{2}(t)\,\,2X(t)X(t\,+\,\tau )]\,+\,E[X^{2}(t\,+\,\tau )]\,\ge \,0 , $$$$ 2R_{X}(0)\,\,2R_{X}(\tau )\,\ge \,0\ \ \Rightarrow \ \ R_{X}(0)\,\ge \,R_{X}(\tau ) . $$  (6)
Symmetry: \(R_{X}(\tau )\,=\,R_{X}(\tau )\);
In order to prove this property, it is sufficient to use the definition \(R_{X}(\tau )\,=\,E[X(t)X(t\,\,\tau )].\)
Letting \(t  \tau \,=\,\sigma \ \ \Rightarrow \ \ t\,=\,\sigma \,+\,\tau \)
$$ R_{X}(\tau )\,=\,E[X(\sigma \,+\,\tau )\cdot X(\sigma )]\,=\,R_{X}(\tau ). $$  (7)
\(E[X(t)]\,=\,\sqrt{R_{X}(\infty )}\) (Signal mean value).
The Power Spectral Density
Properties of the Power Spectral Density
In the sequel, a few properties of the power spectral density function are listed.
 The area under the curve of the power spectral density is equal to the total power of the random process, i.e.,This fact can be verified directly as$$\begin{aligned} P_{X} = \frac{1}{2 \pi } \int _{ \infty }^{+ \infty } S_{X}(\omega ) d \omega . \end{aligned}$$(2.30)in which \(\omega \,=\,2\pi f\).$$\begin{aligned} P_X= & {} R_{X}(0)\, = \, \frac{1}{2\pi }\, \int _{\infty }^{\infty }\, S_{X}(\omega )e^{j\omega 0}\,d\omega \\= & {} \frac{1}{2\pi }\,\int _{\infty }^{\infty }\,S_{X}(\omega )\,d\omega \,=\,\int _{\infty }^{\infty }\,S_{X}(f)\,df, \end{aligned}$$

\(S_{X}(0)\,=\,\int _{\infty }^{\infty }\,R_{X}(\tau )\,d\tau \);
Application: Fig. 2.24 illustrates the fact that the area under the curve of the autocorrelation function is the value of the PSD at the origin.
 If \(R_{X}(\tau )\) is real and even theni.e., \(S_{X}(\omega )\) is real and even.$$\begin{aligned} S_{X}(\omega )= & {} \int _{\infty }^{\infty }\,R_{X}(\tau )[\cos \omega \tau \,\,j\mathrm{sin}\,\omega \tau ]\,d\tau , \nonumber \\= & {} \int _{\infty }^{\infty }\,R_{X}(\tau ) \cos \omega \tau \,d\tau , \end{aligned}$$(2.31)

\(S_{X}(\omega )\ge 0\), since the density reflects a power measure.
 The following identities hold:$$\begin{aligned} \int _{ \infty }^{+ \infty } R_{X}(\tau ) R_{Y}(\tau ) d \tau = \frac{1}{2 \pi } \int _{ \infty }^{+ \infty } S_{X}(\omega ) S_{Y}(\omega ) d \omega . \end{aligned}$$(2.32)$$\begin{aligned} \int _{ \infty }^{+ \infty } R_{X}^{2}(\tau ) d \tau = \frac{1}{2 \pi } \int _{ \infty }^{+ \infty } S_{X}^{2}(\omega ) d \omega . \end{aligned}$$(2.33)
2.5 Linear Systems
The Response of Linear Systems to Random Signals
The computation of the autocorrelation of the output signal, given the autocorrelation of the input signal to a linear system can be performed as follows.
Observation: Preemphasis circuits are used in FM modulators to compensate for the effect of square noise.
The autocorrelation is a special measure of average behavior of a signal. Consequently, it is not always possible to recover a signal from its autocorrelation. Since the power spectral density is a function of the autocorrelation, it also follows that signal recovery from its PSD is not always possible because phase information about the signal has been lost in the averaging operation involved. However, the crosspower spectral densities, relating input–output and output–input, preserve signalphase information and can be used to recover the phase function explicitly.
2.6 Mathematical Formulation for the Digital Signal
This section presents a mathematical formulation for the digital signal, including the computation of the autocorrelation function and the power spectrum density.
2.6.1 Autocorrelation for the Digital Signal
2.6.2 Power Spectrum Density for the Digital Signal
Therefore, the signal design involves pulse shaping as well as the control of the correlation between the transmitted symbols, which can be obtained by signal processing.
The signal bandwidth can be defined in several ways. The most common is the halfpower bandwidth (\(\omega _{3dB}\)). This bandwidth is computed by taking the maximum of the power spectrum density, dividing by two, and finding the frequency for which this value occurs.
2.7 Problems
 (1)Given two events A and B, under which conditions are the following relations true?
 a.
\(A \cap B = \Omega \)
 b.
\(A \cup B = \Omega \)
 c.
\(A \cap B = \bar{A} \)
 d.
\(A \cup B = \emptyset \)
 e.
\(A \cup B = A \cap B \)
 a.
 (2)
If A, B, and C are arbitrary events in a sample space \(\Omega \), express \(A \cup B \cup C\) as the union of three disjoint sets.
 (3)
Show that \(P\{A \cap B \} \le P\{A \} \le P\{A \cup B \} \le P\{A \} + P\{B \}\) and specify the conditions for which equality holds.
 (4)
If \(P\{A \} = a\), \(P\{B \} = b\) and \(P\{ A \cap B \} = ab\), find \(P\{ A \cap \bar{B} \}\) and \(P\{ \bar{A} \cap \bar{B} \}\).
 (5)
Given that A, B, and C are events in a given random experiment, show that the probability that exactly one of the events A, B, or C occurs is \( P\{A \} + P\{B \} + P\{C\}  2P\{ A \cap B \}  2P\{B \cap C \}  2P\{ A \cap C \} + 3P\{A \cap B \cap C \}.\) Illustrate the solution with a Venn diagram.
 (6)
Prove that a set with N elements has \(2^N\) subsets.
 (7)Let A, B, and C be arbitrary events in a sample space \(\Omega \), each one with a nonzero probability. Show that the sample space \(\Omega \), conditioned on A, provides a valid probability measure by proving the following:
 a.
\(P\{ \Omega  A \} = 1\),
 b.
\(P\{ BA \} \le P\{ CA \}, \ \mathrm{if} \ B \subset C \),
 c.
\(P\{ BA \} + P\{ CA \} = P\{ B \cup C / A \} \ \mathrm{if} \ B \cap C = \emptyset \).
Show also that \(P\{ BA \} + P\{ \bar{B}A \} = 1\).
 a.
 (8)Show that if \(A_1, A_2, \dots , A_N\) are independent events then$$ P\{A_1 \cup A_2 \dots \cup A_N \} = 1  ( 1 P\{ A_1 \} ) ( 1 P\{ A_2 \} ) \dots ( 1 P\{ A_n \} ). $$
 (9)
Show that if A and B are independent events then A and \(\bar{B}\) are also independent events.
 (10)
Consider the events \(A_1, A_2, \dots , A_N\) belonging to the sample space \(\Omega \). If \(\sum _{i=1}^{n} P\{A_i\} = 1\), for which conditions \(\bigcup _{i=1}^{n} A_i = \Omega \)? If \(A_1, A_2, \dots , A_N\) are independent and \(P\{ A_i \} = \theta , \ i = 1, \dots , n\), find an expression for \(P \{ \bigcup _{i=1}^{n} A_i \}\).
 (11)
The sample space \(\Omega \) consists of the interval [0, 1]. If sets represented by equal lengths in the interval are equally likely, find the conditions for which two events are statistically independent.
 (12)Using mathematical induction applied to Kolmogorov’s axioms, for \(A_1, A_2, \ldots , A_n\) mutually exclusive events, prove the following property:$$ P \left[ \bigcup _{k=1}^{n} A_k \right] = \sum _{k=1}^{n} P \left[ A_k \right] , \ \mathrm{for} \ n \ge 2. $$
 (13)
A family of sets \(A_n,\ n=1,2,\ldots \) is called limited above by A, by denoting \( A_n \uparrow A\), if \(A_n \subset A_{n+1}\) and \( \bigcup _{n \ge 1} A_n = A\). Using finite additivity, show that if \( A_n \uparrow A\), then \( P(A_n) \uparrow P(A)\).
 (14)
For the Venn diagram in Fig. 2.35, consider that the elements \(\omega \) are represented by points and are equiprobable, i.e., \( P( \omega ) = \frac{1}{4}\), \(\forall \omega \in \Omega \). Prove that the events A, B, and C are not independent.
 (15)For the r.v. X it is known thatFind \(P_X(x)\), \(p_X(x)\) e \(P \{ X > 1/\mu \}\).$$ P \{ X> t \} = e^{ \mu t} ( \mu t + 1 ), \ \mu> 0, \ t>0. $$
 (16)The ratio between the deviations in the length and width of a substrate has the following pdf:Calculate the value of a and find the CPF of X.$$ p_X(x) = \frac{ a }{ 1 + x^2 }, \ \infty< x < \infty . $$
 (17)
The r.v. X is uniformly distributed in the interval (a, b). Derive an expression for the \(n^{th}\) moments \(E[X^n]\) and \(E[(X  m_X)^n]\), in which \( m_X = E[X]\).
 (18)
For a Poisson r.v. X with parameter \(\lambda \), show that \(P\{ X \ \mathrm{even} \} = \frac{1}{2} ( 1 + e^{2 \lambda } )\).
 (19)
By considering a r.v. with mean \(m_X\) and variance \(\sigma _X^2\), compute the moment expression \(E[(X  C)^2]\), for an arbitrary constant C. For which value of C the moment \(E[(X  C)^2]\) is minimized?
 (20)
An exponential r.v. X, with parameter \(\alpha \), has \(P\{ X \ge x \} = e^{\alpha x}\). Show that \(P\{ X> t + s  X> t \} = P\{ X > s \}\).
 (21)
A communication signal has a normal pdf with zero mean and variance \(\sigma ^2\). Design a compressor/expansor for the ideal quantizer for this distribution.
 (22)A voice signal with a Gamma bilateral amplitude probability distribution, given below, is fed through a compressor obeying the \(\mu \)Law of ITUT. Compute the pdf of the signal at the compressor output and sketch an input versus output diagram for \(\gamma = 1\) and \(\alpha = 1/2, 1\) and 2.$$ p_X(x) = \frac{ \gamma ^{ \alpha } }{ \Gamma ( \alpha ) } x^{ \alpha  1 } e^{   \gamma x  }, \ \alpha , \ \gamma > 0. $$
 (23)
For a given exponential probability distribution with parameter \(\alpha \), compute the probability that the r.v. will take values exceeding \(2/\alpha \). Estimate this probability using Tchebychev’s inequality.
 (24)
Calculate all the moments of a Gaussian distribution.
 (25)Show, for the Binomial distributionthat its characteristic function is given by$$ p_X(x) = \sum _{k=0}^{\infty } \left( \begin{array}{c} N \\ k \end{array} \right) p^k (1p)^{Nk} \delta (x  k) $$$$ P_X(\omega ) = \left[ 1  p + p e^{j \omega } \right] ^N. $$
 (26)The Erlang distribution has a characteristic function given byShow that \(E[ X ] = N/a\), \(E[ X^2 ] = N(N+1)/a^2\), and \( \sigma _X^2 = N/a^2\).$$ P_X(\omega ) = \left[ \frac{ a }{ a + j \omega } \right] ^N, \ a > 0, \ N = 1, \ 2, \ \dots . $$
 (27)The Weibull distribution is given byCalculate the mean, the second moment, and the variance for this distribution.$$ p_X(x) = ab x^{b1} e^{  a x^b } u(x) . $$
 (28)For the Poisson distributioncalculate the cumulative probability function, \(P_X(x)\), and show that$$ p_X(x) = e^{b} \sum _{k=0}^{\infty } \frac{ b^k }{ k ! } \delta (x  k) . $$$$ P_X(\omega ) = e^{ b ( 1  e^{j\omega } ) }. $$
 (29)Calculate the statistical mean and the second moment of Maxwell’s distributionexploiting the relationship between a Gaussian distribution and its characteristic function:$$ p_Y(y) = \sqrt{ \frac{2}{\pi } } \frac{y^2}{\sigma ^3} e^{  \frac{y^2}{2 \sigma ^2} } u(y), $$$$ p_X(x) = \frac{ 1 }{ \sqrt{ 2 \pi } \sigma } e^{  \frac{x^2}{2 \sigma ^2} } $$$$ P_X(\omega ) = e^{  \frac{\sigma ^2 \omega ^2}{2} }. $$
 (30)Show that, for a nonnegative r.v. X, it is possible to calculate its mean value through the formulaUsing this formula, calculate the mean of the exponential distribution of the previous problem.$$ E[X] = \int _0^{\infty } (1  P_X(x)) dx. $$
 (31)
The dynamic range of a discretetime signal X(n) is defined as \(W = X_{\max }  X_{\min }\). Assuming that the samples of the signal X(n), \(n = 1, 2, \ldots , N\), are identically distributed, calculate the probability distribution of W.
 (32)For the following joint distributioncalculate the marginal distributions, \(p_X(x)\) and \(p_Y(y)\), and show that the conditional probability density function is given by$$ p_{XY}(x,y) = u(x) u(y) x e^{  x ( y + 1) } $$$$ p_{YX}(yx) = u(x) u(y) x e^{  x y }. $$
 (33)
The r.v.’s V and W are defined in terms of X and Y as \(V = X + a Y\) and \(W = X  aY\), in which a is a real number. Determine a as a function of the moments of X and Y, such that V and W are orthogonal.
 (34)Show that \( E[ E[ YX ] ] = E[ Y ]\), in which$$ E[ E[ YX ] ] = \int _{\infty }^{\infty } E[ Yx] p_X(x) dx. $$
 (35)Find the distribution of the r.v. \(Z = X/Y\), assuming that X and Y are statistically independent r.v.’s having an exponential distribution with parameter equal to 1. Use the formula$$ p_Z(z) = \int _{\infty }^{\infty } y p_{XY}(yz,y) dy. $$
 (36)
A random variable Y is defined by the equation \(Y = X + \beta \), in which X is a r.v. with an arbitrary probability distribution and \(\beta \) is a constant. Determine the value of \(\beta \) which minimizes \(E[X^2]\). Use this value and the expression for \(E[(X \pm Y)^2]\) to determine a lower and an upper limit for E[XY].
 (37)Two r.v.’s, X and Y, have the following characteristic functions, respectively:Calculate the statistical mean of the sum \(X+Y\).$$ P_X(\omega ) = \frac{\alpha }{\alpha + j \omega } \ \ \mathrm{and} \ \ P_Y(\omega ) = \frac{\beta }{\beta + j \omega }. $$
 (38)
Determine the probability density function of \(Z = X/Y\). Given that X and Y are r.v.’s with zero mean and Gaussian distribution, show that Z is Cauchy distributed.
 (39)Determine the probability density function of \(Z = X Y\), given that X is a r.v. with zero mean and a Gaussian distribution, and Y is a r.v. with the following distribution:$$ p_Y(y) = \frac{1}{2} [\delta (y+1) + \delta (y1) ]. $$
 (40)
Let X and Y be statistically independent r.v.’s having exponential distribution with parameter \(\alpha = 1\). Show that \(Z = X + Y\) and \(W = X/Y\) are statistically independent and find their respective probability density function and cumulative probability function.
 (41)Show that if X and Y are statistically independent r.v.’s thenSuggestion: sketch the region \(\{ X<Y \}\).$$ P(X < Y) = \int _{\infty }^{\infty } (1  P_Y(x)) p_X(x) dx. $$
 (42)By using the Cauchy–Schwartz inequality, show that$$ P_{XY} (x,y) \le \sqrt{ P_X(x) P_Y(y) }. $$
 (43)Given the joint distributionin which X and Y are nonnegative r.v.’s, determine k, \(p_X(x)\), \(p_Y(x)\), \(p_X(xy)\), \(p_Y(yx)\), and the first and second moments of this distribution.$$ p_{XY} (x,y) = k x y e^{ x^2  y^2}, $$
 (44)
Determine the relationship between the crosscorrelation and the mean of two uncorrelated processes.
 (45)
Design an equipment to measure the crosscorrelation between two signals, employing delay units, multipliers, and integrators.
 (46)Show that the RMS bandwidth of a signal X(t) is given by$$ B_{RMS}^2 = \frac{ 1 }{ R_X(0) } \frac{ d^2 R_X(\tau ) }{ d \tau ^2 }, \ \mathrm{for}\ \tau = 0. $$
 (47)For the complex random processin which \(A_n\) and \(\theta _n\) are statistically independent r.v.’s, \(\theta _n\) is uniformly distributed in the interval \([0,2\pi ]\) and \(n = 1, \ 2, \dots , N\), show that$$ X(t) = \sum _{n=1}^N A_n e^{ j \omega _o t + j \theta _n } $$$$ R_X(\tau ) = E [ X^*(t) X(t + \tau ) ] = e^{ j \omega _o \tau } \sum _{n=1}^N E[ A_n^2 ]. $$
 (48)The process X(t) is stationary with autocorrelation \(R_X(\tau )\). Letand then show that$$ Y = \int _a^{a + T} X(t) dt, T>0, \ a \ \mathrm{real}, $$$$ E[ Y^2 ] = \int _{T}^{T} ( T  \tau ) R_X(\tau ) d \tau . $$
 (49)The processes X(t) and Y(t) are jointly widesense stationary. LetUnder which conditions, in terms of means and correlation functions of X(t) and Y(t), is Z(t) widesense stationary? Applying these conditions, compute the power spectral density of the process Z(t). What power spectral density would result if X(t) and Y(t) were uncorrelated?$$ Z(t) = X(t) \cos { w_c t } + Y(t) \ \mathrm{sin} \ { w_c t }. $$
 (50)For a given complex random process \(Z(t) = X(t) + jY(t)\), show that$$ E[  Z(t) ^2 ] = R_X(0) + R_Y(0). $$
 (51)Considering that the geometric mean of two positive numbers cannot exceed the correspondent arithmetic mean, and that \(E [ ( Y(t + \tau ) + \alpha X(t) )^2 ] \ge 0\), show that$$  R_{XY}(\tau )  \le \frac{1}{2} [ R_X(0) + R_Y(0) ]. $$
 (52)Let X(t) be a widesense stationary process, with mean \(E[X(t)] \ne 0\). Show thatin which \(C_X(\tau )\) is the autocovariance function of X(t).$$ S_{X}(\omega ) = 2 \pi E^2[ X(t) ] \delta (\omega ) + \int _{\infty }^{\infty } C_X(\tau ) e^{  j \omega \tau } d \tau , $$
 (53)Calculate the RMS bandwidthof a modulated signal having the following power spectral density:$$ B_{RMS}^2 = \frac{ \int _{\infty }^{\infty } \omega ^2 S_{X}(\omega ) d \omega }{ \int _{\infty }^{\infty } S_{X}(\omega ) d \omega } $$in which \(p_X( \cdot )\) denotes the probability density function of the signal X(t).$$ S_{X}(\omega ) = \frac{ \pi A^2 }{ 2 \Delta _{FM} } \left[ p_X \left( \frac{ \omega + \omega _c }{ \Delta _{FM} } \right) + p_X \left( \frac{ \omega  \omega _c }{ \Delta _{FM} } \right) \right] , $$
 (54)
The autocorrelation \(R_X(\tau )\) can be seen as a measure of similarity between X(t) e \(X(t+\tau )\). In order to illustrate this point, consider the process \(Y(t) = X(t)  \rho X(t+\tau )\) and determine the value for \(\rho \) which minimizes the mean square value Y(t).
 (55)
Calculate the crosscorrelation between the processes \(U(t) = X(t) + Y(t)\) and \(V(t) = X(t)  Y(t)\), given that X(t) and Y(t) have zero mean and are statistically independent.
 (56)
Determine the probability density function (pdf) for the stochastic process \(X(t) = e^{A t}\), in which A is a uniformly distributed r.v. over the interval \([1,1]\). Analyze whether the process is stationary and compute its autocorrelation. Sketch a few typical representations of the process X(t), for varying A, as well as the resulting pdf.
 (57)A time series X(t) is used to predict \(X(t + \tau )\). Calculate the correlation between the current value and the predicted value, given that
 (a)
The predictor uses only the current value of the series, X(t).
 (b)
The predictor uses the current value X(t) and its derivative, \(X^{\prime }(t)\).
 (a)
 (58)Find the correlation between the processes V(t) and W(t), in whichand$$ V(t) = X \cos \omega _o t  Y \mathrm{sin} \, \omega _o t, $$in which X and Y are statistically independent r.v.’s with zero mean and variance \(\sigma ^2\).$$ W(t) = Y \cos \omega _o t + X \mathrm{sin} \, \omega _o t, $$
 (59)Calculate the power spectral density for a signal with autocorrelation given by$$ R(\tau ) = A e^{  \alpha \tau  } (1 + \alpha \tau  + \frac{1}{3} \alpha ^2 \tau ^2 ). $$
 (60)Determine the power spectral density of a signal with autocorrelation given by the expression$$ R(\tau ) = A e^{  \alpha \tau  } (1 + \alpha \tau   2 \alpha ^2 \tau ^2 + \frac{1}{3} \alpha ^3 \tau ^3 ). $$
 (61)
Prove the following properties of narrowband random stationary processes.
 (a)
\(S_{XY}( \omega ) = S_{YX}( \omega )\);
 (b)
\(\mathrm{Re} [ S_{XY}( \omega ) ] \ \mathrm{and } \ \mathrm{Re} [ S_{YX}( \omega ) ] \ \mathrm{both \ even}\);
 (c)
\(\mathrm{Im} [ S_{XY}( \omega ) ] \ \mathrm{and } \ \mathrm{Im} [ S_{YX}( \omega ) ] \ \mathrm{both \ odd}\);
 (d)
\(S_{XY}( \omega ) = S_{YX}( \omega ) = 0, \ \mathrm{if} \ X(t) \ \mathrm{and} \ Y(t) \ \mathrm{are \ orthogonal}.\)
 (a)
 (62)Show the following property, for uncorrelated X(t) and Y(t) in a narrowband process,$$ S_{XY}( \omega ) = S_{YX}( \omega ) = 2 \pi E[ X(t) ] E[ Y(t) ] \delta (\omega ). $$
 (63)Prove that the RMS bandwidth of a stochastic signal x(t) is given by$$ B_{RMS}^2 = \frac{ 1 }{ R_X(0) } \frac{ d^2 R_X(\tau ) }{ d \tau ^2 }, \ \mathrm{for}\ \tau = 0. $$
 (64)For the stochastic processesand$$ X(t) = Z(t) \cos (w_c t + \theta ) $$in which \(A, \ \omega _c > 0\), and \(\theta \) is a uniformly distributed r.v. over the interval \([0,2\pi ]\), and statistically independent of Z(t), show that$$ Y(t) = Z(t) \ \mathrm{sin} (w_c t + \theta ), $$$$ S_{XY}( \omega ) = \frac{\pi A}{2} E[ Z(t) ] [ \delta ( w  w_c ) + \delta ( w + w_c ) ]. $$
 (65)Calculate the bandwidthof the noise for a system with the following transfer function:$$ B_{N} = \frac{ 1 }{ H(0)^2 } \int _{0}^{\infty }  H(\omega ) ^2 d \omega $$$$  H(\omega ) ^2 = \frac{ 1 }{ 1 + (\omega / W)^2 }. $$
 (66)
The control system illustrated in Fig. 2.36 has transfer function \(H(\omega )\).
It is known that the Wiener filter, which minimizes the mean square error, and is called optimum in this sense, has a transfer function given byin which \(S_X(\omega )\) denotes the power spectral density of the desired signal and \(S_N(\omega )\) denotes the noise power spectral density. Determine \(H_1(\omega )\) and \(H_2(\omega )\) such that this system operates as a Wiener filter.$$ G(\omega ) = \frac{ S_{X}(\omega ) }{ S_{X}(\omega ) + S_{N}(\omega ) }, $$  (67)Calculate the correlation between the input signal and the output signal for the filter with transfer functiongiven that the input has autocorrelation \(R_X(\tau ) = \delta (\tau )\). Relate the input points for which input and output are orthogonal. Are the signals uncorrelated at these points? Explain.$$ H(\omega ) = u(\omega + \omega _M)  u(\omega + \omega _M), $$
 (68)
Consider the signal \(Y_n = X_n  \alpha X_{n1}\), generated by means of white noise \(X_n\), with zero mean and variance \(\sigma _X^2\). Compute the mean value, the autocorrelation function, and the power spectral density of the signal \(Y_n\).
 (69)Compute the transfer functionof the optimum filter for estimating Y(t) from \(X(t) = Y(t) + N(t)\), in which Y(t) and N(t) are zero mean statistically independent processes.$$ H(\omega ) = \frac{ S_{YX}(\omega ) }{ S_{X}(\omega ) } $$
 (70)
Consider the autoregressive process with moving average \(Y_n + \alpha Y_{n1} = X_{n} + \beta X_{n1}\), built from white noise \(X_n\), with zero mean and variance \(\sigma _X^2\). Calculate the mean value, the autocorrelation, and the power spectral density of the signal \(Y_n\).
 (71)When estimating a time series, the following result was obtained by minimizing the mean square error:in which \(R(\tau )\) denotes the autocorrelation of the process \(\theta (t)\). By simplifying the above expression, show that the increment \(\Delta \theta (t) = \theta (t + \tau )  \theta (t)\) can be written as$$ \theta (t + \tau ) \approx \frac{ R(\tau ) }{R(0)} \theta (t) + \frac{ R^{\prime }(\tau ) }{R^{\prime \prime }(0)} \theta ^{\prime }(t) , $$in which \(\omega _M\) denotes the RMS bandwidth of the process. Justify the intermediate steps used in your proof.$$ \Delta \theta (t) = \tau \theta ^{\prime } (t)  \frac{ (\tau \omega _M )^2 }{2} \theta (t), $$
 (72)
Show that the system below, having \(h(t) = \frac{1}{T} [ u(t)  u(tT) ]\), works as a meter for the autocorrelation of the signal X(t). Is this measure biased?
 (73)
Calculate the transfer function of the optimum filter, for estimating Y(t) from \(X(t) = Y(t) + N(t)\), in which Y(t) and N(t) are zero meanindependent random processes.
 (74)The stochastic process \(Z(t) = X(t) X'(t)\) is built from a Gaussian signal X(t) with zero mean and power spectral density \(S_X(\omega )\). Calculate the power spectral density and the autocorrelation of Z(t), knowing that for the Gaussian processConsidering the signal X(t) has a uniform power spectral density \(S_X(\omega ) = S_0\), between \(\omega _m\) and \(\omega _M\), determine the autocorrelation and the power spectral density of Z(t).$$ S_{X^2} (\omega ) = 2 \int _{\infty }^{\infty } S_X(\omega  \phi ) S_X(\phi ) d \phi . $$
 (75)It is desired to design a proportional and derivative (PD) control system to act over a signal with autocorrelationDetermine the optimum estimator, in terms of mean square, and estimate the signal value after an interval of \(\tau = 1/\alpha \) time units, as a function of the values of X(t) and its derivative \(X'(t)\).$$ R_X(\tau ) = \frac{1  \alpha \tau  }{1 + \alpha \tau  }, \ \tau  \le 1, \ \alpha > 0. $$
References
 Blake, I. F. (1987). An introduction to applied probability. Malabar, Florida: Robert E. Krieger Publishing Co.Google Scholar
 Boyer, C. (1974). History of mathematics. São Paulo, Brasil (in Portuguese): Edgard Blucher Publishers Ltd.Google Scholar
 de Souza, J., & Cavalcante. (1996). The presocratic  fragments, doxography and comments. São Paulo, Brasil (Portuguese): Nova Cultural Publishers Ltd.Google Scholar
 Durant, W. (1996). The history of philosophy. São Paulo, Brasil (in Portuguese): Nova Cultural Publishers Ltd.Google Scholar
 Halmos, P. R. (1960). Naive set theory. Princeton, USA: D. Van Nostrand Company Inc.zbMATHGoogle Scholar
 James, B. R. (1981). Probability: An intermediate level course. Rio de Janeiro, Brasil (in Portuguese): Institute of Pure and Applied Mathematics  CNPq.Google Scholar
 Kennedy, R. S. (1969). Fading dispersive communication channels. New York: Wiley Interscience.Google Scholar
 Lecours, M., Chouinard, J.Y., Delisle, G. Y., & Roy, J. (1988). Statistical modeling of the received signal envelope in a mobile radio channel. IEEE Transactions on Vehicular Technology, 37(4), 204–212.CrossRefGoogle Scholar
 LeonGarcia, A. (1989). Probability and random processes for electrical engineering. Reading, Massachusetts: AddisonWesley Publishing Co.zbMATHGoogle Scholar
 Lipschutz, S. (1968). Set theory. Ao Livro Técnico S.A. Publishers: Rio de Janeiro, Brasil (in Portuguese).Google Scholar
 Papoulis, A. (1981). Probability, random variables, and stochastic processes. Tokyo: McGrawHill.zbMATHGoogle Scholar
 Papoulis, A. (1983). Random modulation: A review. IEEE Transactions on Accoustics, Speech and Signal Processing, 31(1), 96–105.CrossRefGoogle Scholar
 Proakis, J. G. (1990). Digital communications. New York: McGrawHill Book Company.zbMATHGoogle Scholar
 Schwartz, M., Bennett, W., & Stein, S. (1966). Communication systems and techniques. New York: McGrawHill.zbMATHGoogle Scholar
 Zumpano, A., de Lima, B., & Borges, N. (2004). A Medida do Acaso. Ciência Hoje, 34(201), 76–77.Google Scholar