Generalization of the Maximum Entropy Concept and Certain ARMA Processes

Abstract

Given (n + 1) consecutive autocorrelations of a stationary discrete-time stochastic process, one interesting question is how to extend this finite sequence so that the power spectral density associated with the resulting infinite sequence of autocorrelations is nonnegative everywhere. It is well known that when the Hermitian Toeplitz matrix generated from the given autocorrelations is positive-definite, the problem has an infinite number of stable solutions and the particular solution that maximizes the entropy functional results in a stable all-pole model of order n. Since maximization of the entropy functional is equivalent to maximization of the minimum mean-square error associated with one-step predictors, in this paper the problem of obtaining admissible extensions that maximize the minimum mean-square error associated with k-step (k ≤ n) predictors, that are compatible with the given autocorrelations, is studied. It is shown here that the resulting spectrum corresponds to that of a stable ARMA(n, k − 1) process. This true generalization of the maximum entropy extensions for a two-step predictor turns out to be a unique ARMA(n, 1) extension, the details of which are presented here.