Estimation of Parameters of Finite Parameter Models

Abstract

For convenience let us assume that {X _m , m = …, − 1, 0, 1,…} is a strictly stationary process whose probability distribution is parameterized by a k-dimensional parameter θ with real components θ₁, …, θ _k . Assume that the finite dimensional distributions of the process {X _m } are either all absolutely continuous with respect to the corresponding Lebesgue measure or are all discrete. Suppose that one can observe X₁, …, X _n . On the basis of these observations one should like to obtain an effective estimate θ _n (X₁, …, X _n ) of the unknown parameter θ⁰. The maximum likelihood estimate $\hat \theta ({X_1}, \ldots ,{X_n}:\theta )$, is obtained by considering the likelihood function

$${L_n}(\theta ) = {L_n}({X_1}, \ldots ,{X_n};\theta )$$

the joint probability density of the potential observations X₁, …, X _n . The maximum likelihood estimate $\hat \theta ({X_1}, \ldots ,{X_n}:\theta )$ is that value of θ maximizing (assuming that such a maximum exists)

$${L_n}({X_1}, \ldots ,{X_n}:\theta )$$