Asymptotic Statistical Results: Theory and Practice

Abstract

The target of this paper is to discuss the existent difference of Asymptotic Theory in Statistics comparing to Mathematics. There is a need for a limiting distribution in Statistics, usually the Normal one. Adopting the sequential principle the first-order autoregression model and the stochastic approximation are referred for their particular interest for asymptotic results.

Appendix

Let {X _n} be a stochastic process such that the joint distribution of (X ₁, X ₂, …, X _n) has a strictly positive continuous density. The sequence {X _n} will be called absolute fair if for n = 1, 2, ….

The sequence {Y _n}, with Y _n = X ₁ + X ₂ + … + X _n + c is a martingale if

provided that {X _n} is absolutely fair.

Now,

Let $\mathscr {A}_n$ be the σ-algebra generated by (Y ₁, Y ₂, …, Y _n), as above, then

$$\displaystyle \begin{aligned} E(Y_{n+1}|\mathscr{A}_n)=Y_n \end{aligned}$$

from the martingale definition. We replace σ-algebra $\mathscr {A}_n$ by a larger σ-algebra $\mathscr {B}_n$: generated by (Y ₁, Y ₂, …, Y _n and additional random variables depending on the past). The so create process $\mathscr {B}_n$ (containing the past history of the process) is an increasing sequence, i.e. $\mathscr {B}_1\subset \mathscr {B}_2\subset \ldots$. Then the sequence {Y _n} is a martingale with respect to $\mathscr {B}_n$ if and only if

$$\displaystyle \begin{aligned} E(Y_{n+1}|\mathscr{B}_n)=Y_n \end{aligned}$$

Since $\mathscr {B}_n\supset \mathscr {B}_{n-1}$

$$\displaystyle \begin{aligned} E(Y_{n+1}|\mathscr{B}_{n-1})=Y_{n-1} \end{aligned}$$

Now, a move general result helpful in sequential methods is

$$\displaystyle \begin{aligned} E(Y_n|\mathscr{B}_k)=Y_k, k=1,2,\ldots. \end{aligned}$$

That is the condition on knowledge expressed on $\mathscr {B}_k$ provides knowledge for the martingale at the k-th stage.

The Martingale Convergence Theorem provides the mathematical insight for the limiting Normality of the stochastic approximation.

Let {Y _n} be an (infinite) martingale with $E\{Y_n^2\}<c<\infty$, ∀n. Then exists a random variable Y , such that

Furthermore E(Y _n) = E(Y ), ∀n.