Regression Analysis of Stochastic Fatigue Crack Growth Model in a Martingale Difference Framework

Abstract

In the present paper, we are mainly concerned with the degradation mechanism that arises in fatigue crack growth (FCG). The crack evolution mechanism is modeled by a first order stochastic differential system, composed by a deterministic FCG equation perturbed by a stochastic process. The main purpose is to investigate the estimators of the model parameters and establish some asymptotic properties by transforming the initial equation into a regression model. To this purpose, least squares estimation (LSE) is applied in the framework of the errors being martingale differences. The parametric conditional LSE are proved to be consistent. Our results are obtained in the general case and specified for the linear case with focus on a particular application model derived from the most widely used FCG law in fracture mechanics, i.e., the Paris model. We derive the asymptotic normality of the LSE for the nonlinear case with discussion for the linear case. Finally, we provide a numerical example to illustrate the performance of the proposed methodology, on a particular version of the stochastic model.

A Proofs of Sections 3,  5

This section is devoted to the detailed proofs of our results. The previously displayed notation continue to be used in the sequel.

1.1 Proof of Proposition 3.5

Similarly to Theorem 1 of [36], it suffices to show

$$\begin{aligned} \lim _{n \rightarrow \infty }\inf _{|\varvec{\theta }-\varvec{\theta }_{0} |\ge \delta }S_{n}(\varvec{\theta })-S_{n}(\varvec{\theta }_{0})=\infty ~~a.s., \end{aligned}$$

where

$$\begin{aligned} S_{n}(\varvec{\theta })-S_{n}(\varvec{\theta }_{0}) &= \sum _{k=1}^{n}\left( Y_{k}-f_{k}(\varvec{ \theta })\right) ^{2} -\sum _{k=1}^{n}\left( Y_{k}-f_{k}(\varvec{ \theta }_{0})\right) ^{2}\\ &= \sum _{k=1}^{n}d_{k}^{2}(\varvec{\theta }) -2\sum _{k=1}^{n}\eta _{k} d_{k}(\varvec{\theta })\\ & = D_n(\varvec{\theta }, \varvec{\theta _0} )-2L_{n}(\varvec{\theta }) \\ &= D_n(\varvec{\theta }, \varvec{\theta _0} )\left\{ 1-2\frac{L_{n}(\varvec{\theta })}{D_n(\varvec{\theta }, \varvec{\theta _0})}\right\} , \end{aligned}$$

with

$$\begin{aligned} {\left\{ \begin{array}{ll} \displaystyle Y_{k}-f_{k}(\varvec{ \theta })=Y_{k}-f_{k}(\varvec{ \theta }_{0})+f_{k}(\varvec{ \theta }_{0})-f_{k}(\varvec{ \theta })=\eta _{k}-d_{k}(\varvec{\theta }),\\ \displaystyle D_n(\varvec{\theta }, \varvec{\theta _0} )=\sum _{k=1}^n \Big (f_k(\varvec{\theta })-f_k(\varvec{\theta _0})\Big )^2 \text{ and }\\ \displaystyle L_{n}(\varvec{\theta })=\sum _{k=1}^{n} \eta _{k} d_{k}(\varvec{\theta }). \end{array}\right. } \end{aligned}$$

This implies that

$$\begin{aligned} \inf _{\Vert \varvec{\theta }-\varvec{\theta }_{0}\Vert \ge \delta }S_{n}(\varvec{\theta })-S_{n}(\varvec{\theta }_{0})\ge \inf _{\Vert \varvec{\theta }-\varvec{\theta }_{0}\Vert \ge \delta }D_n(\varvec{\theta }, \varvec{\theta _0} )\left\{ 1-2\sup _{\Vert \varvec{\theta }-\varvec{\theta }_{0}\Vert \ge \delta }\frac{|L_{n}(\varvec{\theta })|}{|D_n(\varvec{\theta }, \varvec{\theta _0} )|}\right\} . \end{aligned}$$

By application of Theorem 1 of [74] or Proposition 3.1 of [31], we should show that, for all $\varvec{\theta } \ne \varvec{\theta _0}$,

$$\begin{aligned} D_n(\varvec{\theta }, \varvec{\theta _0} )=\sum _{k=1}^n(f_k(\varvec{\theta })-f_k(\varvec{\theta _0}))^2 \rightarrow \infty , \end{aligned}$$

which follows from Assumption A.3. It suffices to show that

$$\begin{aligned} \lim _{n\rightarrow \infty }\sup _{\Vert \varvec{\theta }-\varvec{\theta }_{0}\Vert \ge \delta }\frac{|L_{n}(\varvec{\theta })|}{|D_n(\varvec{\theta }, \varvec{\theta _0} )|}=0, ~~a.s. \end{aligned}$$

(A.1)

This result exactly follows from the strong law of large numbers for submartingales (Proposition 5.1 of [31]), applied to $d_{k}(\varvec{\theta }), \Vert \varvec{\theta }-\varvec{\theta }_{0}\Vert \ge \delta ,$ by using the assumptions A.1-3. Thus the proof is complete. $\square$

1.2 Proof of Theorem 5.2

Following [74] and the proof of Proposition 6.1 of [31], we infer from the first order Taylor series expansion of $\mathbf { S}_{n}^{\prime }(\widehat{\varvec{\theta }}_{n})$ at $\varvec{\theta }_{0}$ that

$$\begin{aligned} {\mathbf {S}}_{n}^{\prime }(\widehat{\varvec{\theta }}_{n})=\mathbf { S}_{n}^{\prime }(\varvec{\theta }_{0})+\mathbf { S}_{n}^{\prime \prime }(\widetilde{\varvec{\theta }}_{n})\left( \widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}\right) . \end{aligned}$$

(A.2)

Since $\widehat{\varvec{\theta }}_{n}$ is an interior point of $\varvec{\Theta }$ eventually $\mathbf { S}_{n}^{\prime }(\widehat{\varvec{\theta }}_{n})=0$. With ${\mathbf {S}}_{n}^{\prime \prime }(\cdot )$ invertible in a neighborhood of $\varvec{\theta }_{0}$, assumption B.1.iii, Eq. (A.2) implies

$$\begin{aligned} \left( \widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}\right) =-\left[ \mathbf { S}_{n}^{\prime \prime }(\widetilde{\varvec{\theta }}_{n})\right] ^{-1}\mathbf { S}_{n}^{\prime }(\varvec{\theta }_{0}), \end{aligned}$$

and also

$$\begin{aligned} \left( \widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}\right) =-\mathbf { M}_{n}\left[ \mathbf { S}_{n}^{\prime \prime }(\widetilde{\varvec{\theta }}_{n})\mathbf { M}_{n}\right] ^{-1}{\mathbf {S}}_{n}^{\prime }(\varvec{\theta }_{0}). \end{aligned}$$

Hence for any $p\times p$ matrix $\Psi _{n}$, such that (A.1) in Jacob [31] is satisfied, we have

$$\begin{aligned} \Psi _{n}\left( \widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}\right) =-\Psi _{n}\left[ \mathbf { S}_{n}^{\prime \prime }(\widetilde{\varvec{\theta }}_{n})\right] ^{-1}\mathbf { S}_{n}^{\prime }(\varvec{\theta }_{0}), \end{aligned}$$

and also

$$\begin{aligned} \Psi _{n}\left( \widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}\right) =-\Psi _{n}\mathbf { M}_{n}\left[ \mathbf { S}_{n}^{\prime \prime }(\widetilde{\varvec{\theta }}_{n})\mathbf { M}_{n}\right] ^{-1}{\mathbf {S}}_{n}^{\prime }(\varvec{\theta }_{0}). \end{aligned}$$

(A.3)

From

$$\begin{aligned} \mathbf {S}_{n}(\varvec{\theta })=\sum _{k=1}^{n}\left( f_k(\varvec{ \theta }_{0})+ \eta _k-f_{k}(\varvec{ \theta })\right) ^{2}, \end{aligned}$$

we obtain its first derivative

$$\begin{aligned} \mathbf { S}_{n}^{\prime }(\varvec{\theta })=-2\sum _{k=1}^{n}(f_{k}(\varvec{ \theta }_{0})+\eta _{k}-f_{k}(\varvec{ \theta }))\mathbf { f}^{\prime }_{k}(\varvec{\theta }), \end{aligned}$$

(A.4)

and then, the second derivative

$$\begin{aligned} {\mathbf {S}}_{n}^{\prime \prime }(\varvec{\theta })= 2\sum _{k=1}^{n}{\mathbf {f}}^{\prime }_{k}(\varvec{\theta })\mathbf { f}^{\prime }_{k}(\varvec{\theta })^{\top }-2\sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta })-2\sum _{k=1}^{n}(f_{k}(\varvec{ \theta }_{0})-f_{k}(\varvec{ \theta }))\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }). \end{aligned}$$

(A.5)

So, from (A.4), $\mathbf { S}_{n}^{\prime }({\varvec{\theta }})$ at $\varvec{\theta }_{0}$ takes the value

$$\begin{aligned} \mathbf { S}_{n}^{\prime }(\varvec{\theta }_{0})=-2\sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime }_{k}(\varvec{\theta }_{0}). \end{aligned}$$

(A.6)

Notice that the following terms

• 1. $\displaystyle \left[ \sum _{k=1}^{n}\mathbf { f}^{\prime }_{k}(\widetilde{\varvec{\theta }}_{n})\mathbf { f}^{\prime }_{k}(\widetilde{\varvec{\theta }}_{n})^{\top }\right] {\mathbf {M}}_{n} {\mathop \rightarrow \limits ^{{\mathbb {P}}}} {\mathbf {I}} ~~\text{ according } \text{ to } \text{ assumption } \text{ B.1.(iii) },$

• 2. $\displaystyle \left[ \sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime \prime }_{k}(\widetilde{\varvec{\theta }}_{n})\right] {\mathbf {M}}_{n} {\mathop \rightarrow \limits ^{{\mathbb {P}}}} 0 , ~~\text{ according } \text{ to } \text{ assumption } \text{ B.1.(i) },$

• 3. $\displaystyle \left[ \sum _{k=1}^{n}\Big (f_{k}(\varvec{ \theta }_{0})-f_{k}(\widetilde{\varvec{\theta }}_{n})\Big )\mathbf { f}^{\prime \prime }_{k}(\widetilde{\varvec{\theta }}_{n})\right] {\mathbf {M}}_{n} {\mathop \rightarrow \limits ^{{\mathbb {P}}}} 0 ~~\text{ according } \text{ to } \text{ assumption } \text{ B.1.(ii) }.$

Let us define the matrix ${\varvec{K}}_n$ by

$$\begin{aligned} {\varvec{K}}_n=\frac{1}{2}\mathbf { S}_{n}^{\prime \prime }(\widetilde{\varvec{\theta }}_{n})\mathbf { M}_{n}= & {} \left[ \sum _{k=1}^{n}\mathbf { f}^{\prime }_{k}(\widetilde{\varvec{\theta }}_{n})\mathbf { f}^{\prime }_{k}(\widetilde{\varvec{\theta }}_{n})^{\top }\right] {\mathbf {M}}_{n}-\left[ \sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime \prime }_{k}(\widetilde{\varvec{\theta }}_{n})\right] {\mathbf {M}}_{n}\\&-\left[ \sum _{k=1}^{n}(f_{k}(\varvec{ \theta }_{0})-f_{k}(\widetilde{\varvec{\theta }}_{n}))\mathbf { f}^{\prime \prime }_{k}(\widetilde{\varvec{\theta }}_{n})\right] {\mathbf {M}}_{n}. \end{aligned}$$

We infer that we have

$$\begin{aligned} \lim _{n\rightarrow \infty }{\varvec{K}}_n{\mathop {=}\limits ^{{\mathbb {P}}}}{\mathbf {I}}. \end{aligned}$$

An application of Slutsky’s theorem in the Eq. (A.3), since, as $n\rightarrow \infty$,

$$\begin{aligned} \left[ \mathbf {S}_{n}^{\prime \prime }(\widetilde{\varvec{\theta }}_{n})\mathbf { M}_{n}\right] ^{-1} {\mathop \rightarrow \limits ^{{\mathbb {P}}}} \left[ \frac{1}{2}{\mathbf {I}}^{-1}\right] , \end{aligned}$$

implies that

$$\begin{aligned} \lim _{n\rightarrow \infty }\Psi _{n}\left( \widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}\right) {\mathop = \limits ^{d}}\lim _{n\rightarrow \infty }\left\{ -\Psi _{n}{\mathbf {M}}_{n}\left[ \frac{1}{2}\mathbf { I}^{-1} \right] \mathbf { S}_{n}^{\prime }(\varvec{\theta }_{0})\right\} . \end{aligned}$$

(A.7)

Then we have

$$\begin{aligned} \lim _{n\rightarrow \infty }\Psi _{n}\left( \widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}\right) {\mathop {=}\limits ^{d}}-\lim _{n\rightarrow \infty }\frac{1}{2}\Psi _{n}{\mathbf {M}}_{n}\mathbf { S}_{n}^{\prime }(\varvec{\theta }_{0}), \end{aligned}$$

(A.8)

and

$$\begin{aligned} \lim _{n}\Psi _{n}\left( \widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}\right) {\mathop =\limits ^{d}}\lim _{n\rightarrow \infty }\Psi _{n}\mathbf { M}_{n}\sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime }_{k}(\varvec{\theta }_{0}), \end{aligned}$$

(A.9)

which from the assumption B.2 that

$$\begin{aligned} \lim _{n\rightarrow \infty } \Psi _{n}\mathbf { M}_{n}\sum _{k=1}^{n}\eta _{k}{\mathbf {f}}^{\prime }_{k}({\varvec{\theta }}_{0}) \end{aligned}$$

exists in distribution, this suffices to complete the proof of Theorem 5.2. $\square$

1.3 Proof of Theorem 6.3

This demonstration follows closely the proof of Theorem 2 of Lai [36]. By (A.4) and since $\mathbf { S}_{n}^{\prime }(\widehat{\varvec{\theta }}_{n})=0$, we have

$$\begin{aligned} 0=\frac{-1}{2}\mathbf { S}_{n}^{\prime }(\widehat{\varvec{\theta }}_{n})=\sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime }_{k}(\widehat{\varvec{\theta }}_{n})-\sum _{k=1}^{n}\mathbf { f}^{\prime }_{k}(\widehat{\varvec{\theta }}_{n})\big (f_{k}(\widehat{\varvec{\theta }}_{n})-f_{k}(\varvec{ \theta }_{0})\big ). \end{aligned}$$

(A.10)

An application of the mean value theorem implies readily that

$$\begin{aligned} \frac{{\mathbf {f}}_{k}^{\prime }(\widehat{\varvec{\theta }}_{n})-{\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})}{\widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}}={\mathbf {f}}^{\prime \prime }_{k}({\varvec{\theta }}_{1}), \end{aligned}$$

which gives

$$\begin{aligned} {{\mathbf {f}}_{k}^{\prime }(\widehat{\varvec{\theta }}_{n})={\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})}+{\mathbf {f}}^{\prime \prime }_{k}({\varvec{\theta }}_{1})({\widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}}), ~~ \text{ for } ~~ {\varvec{\theta }}_{1} \in ({\varvec{\theta }}_{0}, \widehat{\varvec{\theta }}_{n}), \end{aligned}$$

and so

$$\begin{aligned} \sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime }_{k}(\widehat{\varvec{\theta }}_{n})=\sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime }_{k}({\varvec{\theta }}_{0})+ \sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime \prime }_{k}({\varvec{\theta }}_{1})(\widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}). \end{aligned}$$

(A.11)

Again from the mean value theorem, one can see that

$$\begin{aligned} \mathbf {f}_{k}^{\prime }(\widehat{\varvec{\theta }}_{n}) &= \mathbf { f}_{k}^{\prime }(\varvec{ \theta }_{0})+\mathbf { f}^{\prime \prime }_{k}({\varvec{\theta }}_{2})({\widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}}), ~~ \text{ for } ~~ {\varvec{\theta }}_{2} \in ({\varvec{\theta }}_{0}, \widehat{\varvec{\theta }}_{n}), \\ \frac{{\mathbf {f}}_{k}(\widehat{\varvec{\theta }}_{n})-\mathbf { f}_{k}(\varvec{ \theta }_{0})}{\widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}} &= \mathbf { f}^{\prime }_{k}({\varvec{\theta }}_{3})^{\top }, ~~ \text{ for } ~~ {\varvec{\theta }}_{3} \in ({\varvec{\theta }}_{0}, \widehat{\varvec{\theta }}_{n}), \\ {\mathbf {f}}_{k}^{\prime }({\varvec{\theta }}_{3}) &= \mathbf { f}_{k}^{\prime }(\varvec{ \theta }_{0})+\mathbf { f}^{\prime \prime }_{k}({\varvec{\theta }}_{4})({{\varvec{\theta }}_{3}-{\varvec{\theta }}_{0}}), ~~ \text{ for } ~~ {\varvec{\theta }}_{4} \in ({\varvec{\theta }}_{0}, {\varvec{\theta }}_{3}). \end{aligned}$$

We infer that we have

$$\begin{aligned}&\sum _{k=1}^{n}\mathbf { f}^{\prime }_{k}(\widehat{\varvec{\theta }}_{n})\big (\mathbf { f}_{k}(\widehat{\varvec{\theta }}_{n})-\mathbf { f}_{k}(\varvec{ \theta }_{0})\big )\\&\qquad = \sum _{k=1}^{n}\left[ {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})+\mathbf { f}^{\prime \prime }_{k}({\varvec{\theta }}_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{\theta }_{0})\right] \left[ \mathbf { f}_{k}^{\prime }({\varvec{\theta }}_{3})^{\top }(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0})\right] \\&\qquad = \sum _{k=1}^{n}\left[ {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})+\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{\theta }_{0})\right] \left[ {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})+\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_{4})(\varvec{\theta }_{3}-\varvec{\theta }_{0})\right] ^{\top }(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}) \\&\qquad = \sum _{k=1}^{n}\left[ {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})+{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{\theta }_{0})\right] \left[ {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})^{\top }+( \varvec{\theta }_{3}-\varvec{\theta }_{0})^{\top }{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_{4})^{\top }\right] (\widehat{\varvec{\theta }}_{n}-\varvec{\theta }_{0}) \\&\qquad = \left[ \sum _{k=1}^{n} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) ^{\top }+\sum _{k=1}^{n}{\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})(\varvec{\theta }_{3}-\varvec{ \theta }_{0})^{\top } {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) ^{\top }\right. \\&\left. \qquad +\,\sum _{k=1}^{n}\mathbf { f}_{k}^{\prime \prime }(\varvec{ \theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) ^{\top }+\sum _{k=1}^{n}{\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0})(\varvec{\theta }_{3}-\varvec{ \theta }_{0})^{\top } {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) ^{\top }\right] (\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}). \end{aligned}$$

It follows that

$$\begin{aligned} \sum _{k=1}^{n}\eta _{k}{\mathbf {f}}^{\prime }_{k}(\varvec{\theta }_{0})= & {} \left[ \sum _{k=1}^{n} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) ^{\top }+\sum _{k=1}^{n}{\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})(\varvec{\theta }_{3}-\varvec{ \theta }_{0})^{\top } {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) ^{\top }\right. \\&\quad +\sum _{k=1}^{n}{\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) ^{\top }+\sum _{k=1}^{n}{\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0})(\varvec{\theta }_{3}-\varvec{ \theta }_{0})^{\top } {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) ^{\top }\\&\left. \quad -\sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_{0})+\sum _{k=1}^{n}\eta _{k}\big (\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_{1})-\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_{0})\big )\right] (\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}). \end{aligned}$$

By multiplying by $\varvec{\Lambda }_{n}^{1/2}$, we obtain

$$\begin{aligned} \sum _{k=1}^{n}\eta _{k}{\mathbf {f}}^{\prime }_{k}(\varvec{\theta }_{0})\, = & {} \varvec{\Lambda }_{n}^{1/2}\Big [\varvec{\Lambda }_{n}^{-1/2}\sum _{k=1}^{n} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) ^{\top }\varvec{\Lambda }_{n}^{-1/2}\\&\quad + \varvec{\Lambda }_{n}^{-1/2}\sum _{k=1}^{n}{\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})(\varvec{\theta }_{3}-\varvec{ \theta }_{0})^{\top } {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) ^{\top }\varvec{\Lambda }_{n}^{-1/2}\\&\quad +\varvec{\Lambda }_{n}^{-1/2}\sum _{k=1}^{n}{\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) ^{\top }\varvec{\Lambda }_{n}^{-1/2} \\&\quad + \varvec{\Lambda }_{n}^{-1/2}\sum _{k=1}^{n}{\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0})(\varvec{\theta }_{3}-\varvec{ \theta }_{0})^{\top } {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4})^{\top }\varvec{\Lambda }_{n}^{-1/2}\\&\quad -\varvec{\Lambda }_{n}^{-1/2}\sum _{k=1}^{n}\eta _{k}{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_{0})\varvec{\Lambda }_{n}^{-1/2}\\&\quad + \varvec{\Lambda }_{n}^{-1/2}\sum _{k=1}^{n}\eta _{k}\big (\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_{1})-\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_{0})\big )\varvec{\Lambda }_{n}^{-1/2}\Big ]\varvec{\Lambda }_{n}^{1/2}(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}). \end{aligned}$$

Then, we evaluate the preceding terms one by one, for all large n. It will be shown that all terms tend to 0 in probability, except for the first one, which appears in the central limit theorem, (Eq. 6.9). We work with suitably chosen Hilbert spaces H and by making use of martingale central limit theorems and probability bounds for H-valued martingales. Recall that $\{\eta _k, {\mathscr {F}}_{k-1}, k\ge 1 \}$ are martingale differences and $\{\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta })-\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_0)\}$ are ${\mathscr {F}}_{k-1}$ measurable H-valued random variables. The last term of the previous equation gives

$$\begin{aligned}&\Big |\Big | \varvec{\Lambda }_{n}^{-1/2} \sum _{k=1}^{n}\eta _{k}({\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_{1})-{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_{0})\varvec{\Lambda }_{n}^{-1/2}\Big |\Big |^2 \\&\quad \le \Big |\Big | \varvec{\Lambda }_{n}^{-1/2}\Big |\Big |^4 p^2 \sup _{\varvec{\theta }\in {\varvec{B}}(\varvec{\theta }_0)}\Big |\sum _{k=1}^n \eta _k\{{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta })-{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_0)\}\Big | ^2 \\&\quad \le \Big |\Big | \varvec{\Lambda }_{n}^{-1/2}\Big |\Big |^4 p^2 vol({\varvec{B}}) \Big |\Big |\sum _{k=1}^n \eta _k\{{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta })-{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_0)\} \Big |\Big |^2 \\&\quad \le p^2 \sum _{m=1}^p \sum _{{\varvec{j}}\in {\varvec{J}}(m, p)}O_{{\mathbb {P}}}\left( \sum _{k=1}^n \int _{{\varvec{B}}(\varvec{\theta }_0)} [\varvec{D_j}({\mathbf {f}}^{\prime \prime }_{k})]^2 d {\theta }_{j_1} \cdots d {\theta }_{j_m}\right) /\lambda _{\min }^4(\varvec{\Lambda }_{n}^{1/2}) \\&\quad {\mathop {\rightarrow }\limits ^{{\mathbb {P}}}} 0, \end{aligned}$$

with

$$\begin{aligned} \Big |\Big | \varvec{\Lambda }_{n}^{-1/2}\Big |\Big | = 1 / \lambda _{\min }(\varvec{\Lambda }_{n}^{1/2}), \end{aligned}$$

where we have used assumption C.4 for the last equation.

In more detail now, it is important to see here that the sup-norm of

$$\begin{aligned} \sum _{k=1}^n \eta _k\{\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta })-\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_0)\} \end{aligned}$$

gets bounded by its H-norm, with the use of Cauchy-Schwarz inequality. Since

$$\begin{aligned} {\mathbf {f}}_{k}(\varvec{\theta })-\mathbf { f}_{k}(\varvec{\theta }_0)=\sum _{m=1}^p \sum _{{\varvec{j}} \in {\varvec{J}}(m, p)} \int _{{\theta }_{0_{j_m}}}^{{\theta }_{j_m}} \cdots \int _{{\theta }_{0_{j_1}}}^{{\theta }_{j_1}} \varvec{D_j}{\mathbf {f}}_k d {\theta }_{j_1} \cdots d {\theta }_{j_m}, \end{aligned}$$

this implies

$$\begin{aligned} {\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta })-{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_0)=\sum _{m=1}^p \sum _{{\varvec{j}} \in {\varvec{J}}(m, p)} \int _{{\theta }_{0_{j_m}}}^{{\theta }_{j_m}} \cdots \int _{{\theta }_{0_{j_1}}}^{{\theta }_{j_1}} \varvec{D_j}{\mathbf {f}}^{\prime \prime }_k d {\theta }_{j_1} \cdots d {\theta }_{j_m}. \end{aligned}$$

Making use of the Cauchy–Schwarz inequality, we infer that

$$\begin{aligned}&\sup _{\varvec{\theta }\in {\varvec{B}}(\varvec{\theta }_0)}\Big |\sum _{k=1}^n \eta _k\{{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta })-\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_0)\}\Big | ^2\\&\quad = \sup _{\varvec{\theta }\in {\varvec{B}}(\varvec{\theta }_0)}\Big |\sum _{m=1}^p \sum _{{\varvec{j}}} \int _{{\theta }_{0_{j_m}}}^{{\theta }_{j_m}} \cdots \int _{{\theta }_{0_{j_1}}}^{{\theta }_{j_1}}\sum _{k=1}^n \eta _k \varvec{D_j}{\mathbf {f}}^{\prime \prime }_k d {\theta }_{j_1} \cdots d {\theta }_{j_m}\Big | ^2 \\&\quad \le \left\{ \int _{{\theta }_{0_{j_m}}}^{{\theta }_{j_m}} \cdots \int _{{\theta }_{0_{j_1}}}^{{\theta }_{j_1}} d {\theta }_{j_1} \cdots d {\theta }_{j_m} \right\} \\&\qquad \times \left\{ \int _{{\theta }_{0_{j_m}}}^{{\theta }_{j_m}} \cdots \int _{{\theta }_{0_{j_1}}}^{{\theta }_{j_1}} \left[ \sum _{k=1}^n \eta _k \varvec{D_j}{\mathbf {f}}^{\prime \prime }_k \right] ^2 d {\theta }_{j_1} \cdots d {\theta }_{j_m}\right\} \\&\quad =vol({\varvec{B}}\big (\varvec{\theta }_{0})\big ) \left\| \sum _{k=1}^n \eta _k\{\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta })-\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_0)\} \right\| ^2_{H}. \end{aligned}$$

We obtain

$$\begin{aligned} \left\| \sum _{k=1}^n \eta _k\{\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta })-\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_0)\} \right\| ^2_{H}= O_{{\mathbb {P}}}\left( \sum _{k=1}^n {\mathbb {E}}\Big ( \left\| \eta _k\{\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta })-\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_0)\} \right\| ^2_{H} | {\mathscr {F}}_{k-1} \Big )\right) . \end{aligned}$$

(A.12)

For nonrandom constant $c_k$ sufficiently large so that

$$\begin{aligned} {\mathbb {P}}\Big \{ \left\| \mathbf { f}^{\prime \prime }_{k}(\varvec{\theta })-\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_0) \right\| ^2_{H}+ {\mathbb {E}}(\eta _k^2| {\mathscr {F}}_{k-1} ) > c_k \Big \} \le k^{-2}. \end{aligned}$$

By Borel-Cantelli lemma, we have

$$\begin{aligned}&{\mathbb {P}}\Big (\eta _k\{{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta })-{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_0)\}\mathbb {1} \Big \{ \left\| {\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta })-{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_0) \right\| ^2_{H} \le c_k ~ \text{ and } {\mathbb {E}}( \eta _k^2| {\mathscr {F}}_{k-1} )\le c_k \Big \} \nonumber \\&\quad =\eta _k\{\mathbf {f}^{\prime \prime }_{k}(\varvec{\theta })-\mathbf {f}^{\prime \prime }_{k}(\varvec{\theta }_0)\} \text{ for } \text{ large } k \Big )=1. \end{aligned}$$

(A.13)

We readily infer that

$$\begin{aligned} {\mathbb {P}}\Big ( \left\| \mathbf { f}^{\prime \prime }_{k}(\varvec{\theta })-\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_0) \right\| ^2_{H}\le c_k \text{ and } {\mathbb {E}}( \eta _k^2| {\mathscr {F}}_{k-1} )\le c_k \Big ) = {\mathbb {P}}\Big ( \left\| \mathbf { f}^{\prime \prime }_{k}(\varvec{\theta })-\mathbf { f}^{\prime \prime }_{k}(\varvec{\theta }_0 \right\| ^2 \le c_k\Big ), \end{aligned}$$

provided that

$$\begin{aligned} \sup _k{{\mathbb {E}}(\eta _k^2|{\mathcal {F}}_{k-1}) }< \infty ~~ a.s. \end{aligned}$$

Hence, from (A.12) and (A.13), it follows

$$\begin{aligned} \left\| \sum _{k=1}^n \eta _k\{{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta })-{\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_0)\} \right\| ^2 =O_{{\mathbb {P}}}\left( \sum _{k=1}^n \int _{(\varvec{\theta }_0 ; {\varvec{j}})} [\varvec{D_j}({\mathbf {f}}^{\prime \prime }_{k})]^2 d {\theta }_{j_1} \cdots d {\theta }_{j_m}\right) . \end{aligned}$$

Again, an application of Cauchy-Schwarz inequality gives

$$\begin{aligned} \left\| \varvec{\Lambda }_{n}^{-1/2} {\sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime \prime }_{k}({\varvec{\theta }}_{0})}\varvec{\Lambda }_{n}^{-1/2} \right\| ^2 &\le \left\| \varvec{\Lambda }_{n}^{-1/2} \right\| ^4 \left\| {\sum _{k=1}^{n}\eta _{k}{\mathbf {f}}^{\prime \prime }_{k}({\varvec{\theta }}_{0})} \right\| ^2 \\ &= \left\| \varvec{\Lambda }_{n}^{-1/2} \right\| ^4 O_{{\mathbb {P}}}\Big (\sum _{k=1}^n {\mathbf {E}}\Big (\left\| \eta _k\ {\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_0)\right\| ^2 | {\mathscr {F}}_{k-1}\Big )\Big ) \\ &= \Big |\Big | \varvec{\Lambda }_{n}^{-1/2}\Big |\Big |^4 O_{{\mathbb {P}}\mathbb {}}\Big (\sum _{k=1}^n \left\| {\mathbf {f}}^{\prime \prime }_{k}(\varvec{\theta }_0)\right\| ^2 \Big ) \\ &\le \lambda _{min}^2(\varvec{\Lambda }_{n}^{1/2})/ \lambda _{min}^4(\varvec{\Lambda }_{n}^{1/2}) {\mathop \rightarrow \limits ^{{\mathbb {P}}}} 0. \end{aligned}$$

We now treat the following term

$$\begin{aligned}&\left\| \varvec{\Lambda }_{n}^{-1/2} {\sum _{k=1}^{n}{\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})({\varvec{\theta }}_{3}-\varvec{ \theta }_{0})^{\top } {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) ^{\top }}\varvec{\Lambda }_{n}^{-1/2}\right\| ^2 \\&\quad \le \left\| \sum _{k=1}^{n}\left[ \varvec{\Lambda }_{n}^{-1/2} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})\right] \left[ {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) (\varvec{\theta }_{3}-\varvec{ \theta }_{0})\right] ^{\top }\right\| ^2 \left\| \varvec{\Lambda }_{n}^{-1/2}\right\| ^2 \\&\quad \text{ from } \text{ Cauchy{-}Schwarz } \text{ inequality }\\&\quad \le \sum _{k=1}^{n} \left\| \varvec{\Lambda }_{n}^{-1/2} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})\right\| ^2 \sum _{k=1}^{n} \left\| {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) ({\varvec{\theta }}_{3}-\varvec{ \theta }_{0})\right\| ^2 \left\| \varvec{\Lambda }_{n}^{-1/2}\right\| ^2 \\&\quad \le \sum _{k=1}^{n} \left\| \varvec{\Lambda }_{n}^{-1/2} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})\right\| ^2 \sum _{k=1}^{n} \Big \{p^2 \max \Big |{\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) \Big |^2 \left\| (\varvec{\theta }_{3}-\varvec{ \theta }_{0}) \right\| ^2 \Big \} \left\| \varvec{\Lambda }_{n}^{-1/2}\right\| ^2 {\mathop \rightarrow \limits ^{{\mathbb {P}}}} 0. \end{aligned}$$

Let us treat the following term,

$$\begin{aligned}&\left\| \varvec{\Lambda }_{n}^{-1/2} \sum _{k=1}^{n}{\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) ^{\top }\varvec{\Lambda }_{n}^{-1/2}\right\| ^2\\&\quad \le \left\| \varvec{\Lambda }_{n}^{-1/2}\right\| ^2 \left\| \sum _{k=1}^{n} \left[ {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{2}) (\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}\right] \left[ \varvec{\Lambda }_{n}^{-1/2} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})\right] ^{\top }\right\| ^2 \\&\quad ~~~~\text{(applying } \text{ Cauchy{-}Schwarz } \text{ inequality) }\\&\quad \le \sum _{k=1}^{n} \left\| \varvec{\Lambda }_{n}^{-1/2} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})\right\| ^2 \sum _{k=1}^{n} \left\| {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) ({\varvec{\theta }}_{3}-\varvec{ \theta }_{0})\right\| ^2 \left\| \varvec{\Lambda }_{n}^{-1/2}\right\| ^2 \\&\quad \le \sum _{k=1}^{n} \left\| \varvec{\Lambda }_{n}^{-1/2} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})\right\| ^2 \sum _{k=1}^{n} \Big \{p^2 \max \Big |{\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{2}) \Big |^2 \left\| ({\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0})}\right\| ^2\Big \} \left\| \varvec{\Lambda }_{n}^{-1/2}\right\| ^2 {\mathop \rightarrow \limits ^{{\mathbb {P}}}} 0. \end{aligned}$$

Once more, an application of Cauchy–Schwarz inequality implies that

$$\begin{aligned}&\left\| \varvec{\Lambda }_{n}^{-1/2} \sum _{k=1}^{n}\mathbf { f}_{k}^{\prime \prime }(\varvec{ \theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0})(\varvec{\theta }_{3}-\varvec{ \theta }_{0})^{\top } {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) ^{\top } \varvec{\Lambda }_{n}^{-1/2}\right\| ^2 \\&\quad \le \left\| \varvec{\Lambda }_{n}^{-1/2}\right\| ^{4} \left\| \sum _{k=1}^{n}{\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0})(\varvec{\theta }_{3}-\varvec{ \theta }_{0})^{\top } {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) ^{\top } \right\| ^2 \\&\quad \le \left\| \varvec{\Lambda }_{n}^{-1/2}\right\| ^{4} \sum _{k=1}^{n}\left\| {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{2})(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0})\right\| ^2 \sum _{k=1}^{n} \left\| {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) (\varvec{\theta }_{3}-\varvec{ \theta }_{0}) \right\| ^2 \\&\quad \le \left\| \varvec{\Lambda }_{n}^{-1/2} \right\| ^{4} \sum _{k=1}^{n} \Big \{p^2 \max \Big | \mathbf { f}_{k}^{\prime \prime }(\varvec{ \theta }_{2}) \Big |^2 \left\| ({\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0})}\right\| ^2\Big \}\\&\qquad \times \sum _{k=1}^{n} \Big \{ p^2 \max \Big | \mathbf { f}_{k}^{\prime \prime }(\varvec{ \theta }_{4}) \Big |^2 \left\| (\varvec{\theta }_{3}-\varvec{ \theta }_{0}) \right\| ^2\Big \} {\mathop \rightarrow \limits ^{{\mathbb {P}}}} 0. \end{aligned}$$

Making use of the Assumption B.1.iii, we infer that

$$\begin{aligned} \varvec{\Lambda }_{n}^{-1/2} \sum _{k=1}^{n} \left\{ {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) ^{\top } \right\} ^{1/2} {\mathop \rightarrow \limits ^{{\mathbb {P}}}} {\mathbf {I}}. \end{aligned}$$

So, it comes out from the above that

$$\begin{aligned} \varvec{\Lambda }_{n}^{-1/2} \sum _{k=1}^{n}\eta _{k}{\mathbf {f}}^{\prime }_{k}(\varvec{\theta }_{0})= O_{{\mathbb {P}}}(1) \left\{ \sum _{k=1}^{n} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) ^{\top }\right\} ^{1/2}\varvec{\Lambda }_{n}^{-1/2}\varvec{\Lambda }_{n}^{1/2}(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}), \end{aligned}$$

or

$$\begin{aligned} \varvec{\Lambda }_{n}^{-1/2} \sum _{k=1}^{n}\eta _{k}\mathbf { f}^{\prime }_{k}(\varvec{\theta }_{0})=O_{{\mathbb {P}}}(1) \left\{ \sum _{k=1}^{n} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) ^{\top } \right\} ^{1/2}(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}). \end{aligned}$$

(A.14)

Following Theorem 2 of [36], the martingale central limit theorem of [26] (in chaptrer 3 and chapter 6) is applied, since the following conditions are satisfied:

1.4 Statement D

• 1. $\max \left\| \varvec{\Lambda }_{n}^{-1/2}\eta _k\mathbf { f}^{\prime }_{k}(\varvec{\theta }_{0}) \right\| {\mathop \rightarrow \limits ^{{\mathbb {P}}}} 0$ , and

  $$\begin{aligned} {\left\{ \begin{array}{ll} \max \left\| \varvec{\Lambda }_{n}^{-1/2}{\mathbf {f}}^{\prime }_{k}(\varvec{\theta }_{0}) \right\| {\mathop \rightarrow \limits ^{{\mathbb {P}}}} 0, \qquad \text{ from } \text{ assumption } \text{ C.2 } \\ \qquad \text{ and }\\ \sup _k {\mathbb {E}} \big (\eta _k^2 | {\mathscr {F}}_{k-1}\big ) < \infty .\\ \end{array}\right. } \end{aligned}$$

• 2. Also

  $$\begin{aligned} \sum _{k=1}^{n} \varvec{\Lambda }_{n}^{-1/2} \eta _k {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) \sum _{k=1}^{n} \left[ \varvec{\Lambda }_{n}^{-1/2} \eta _k {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})\right] ^{\top } {\mathop \rightarrow \limits ^{{\mathbb {P}}}} \sigma ^2 \varvec{\Lambda }_{n}^{-1/2} \sum _{k=1}^{n} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})^{\top } {\varvec{\Lambda }_{n}^{-1/2}}^{\top } . \end{aligned}$$

  By using the fact that

  $$\begin{aligned} \varvec{\Lambda }_{n}^{-1/2}\left( \sum _{k=1}^{n} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})^{\top }\right) ^{1/2} {\mathop \rightarrow \limits ^{{\mathbb {P}}}} {\mathbf {I}}, \end{aligned}$$

  we readily obtain that

  $$\begin{aligned} \sum _{k=1}^{n} \varvec{\Lambda }_{n}^{-1/2} \eta _k {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) \sum _{k=1}^{n} \left[ \varvec{\Lambda }_{n}^{-1/2} \eta _k {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})\right] ^{\top }{\mathop {\rightarrow }\limits ^{{\mathbb {P}}}} \sigma ^2 {\mathbf {I}}. \end{aligned}$$

Consequently, by the preceding conditions and making use of (A.14), we have, as $n\rightarrow \infty$

$$\begin{aligned} \left\{ \sum _{k=1}^{n} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) ^{\top } \right\} ^{1/2}(\widehat{\varvec{\theta }}_{n}-\varvec{ \theta }_{0}) {\mathop {\rightarrow }\limits ^{d}} {\mathcal {N}} (0, \sigma ^2 {\mathbf {I}}). \end{aligned}$$

Now, as in [36], we may consider the assumption that

$$\begin{aligned} \frac{1}{2n}{\mathbf {S}}_n^{\prime \prime }(\varvec{\theta }_0) {\rightarrow } {\mathbf {V}}, ~~~\text{ a.s.}, \end{aligned}$$

(A.15)

where ${\mathbf {V}}$ a positive definite and nonrandom matrix. Because

$$\begin{aligned} \frac{1}{2} {\mathbf {S}}_n^{\prime \prime }(\varvec{\theta }_0)=- \sum _{k=1}^{n} \eta _k {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{0})+\sum _{k=1}^{n} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})^{\top }, \end{aligned}$$

and by the martingale strong law [38], we infer that

$$\begin{aligned} \sum _{k=1}^{n} \eta _k {\mathbf {f}}_{k}^{\prime \prime }(\varvec{ \theta }_{0})= o \left( \sum _{k=1}^{n} \left\| \mathbf { f}_{k}^{\prime \prime }(\varvec{ \theta }_{0})\right\| ^2 \right) + O(1) \quad \text{ a.s.} \end{aligned}$$

(A.16)

So, the assumption (A.15) becomes simpler:

$$\begin{aligned} 1/n \sum _{k=1}^{n} {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}) {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0})^{\top } {\rightarrow } {\mathbf {V}} \end{aligned}$$

(A.17)

Similarly to [26] (Chapter 6), we have as $n\rightarrow \infty$,

$$\begin{aligned} -\frac{1}{{\sqrt{n}}}\sum _{k=1}^{n} \eta _k {\mathbf {f}}_{k}^{\prime }(\varvec{ \theta }_{0}){\mathop \rightarrow \limits ^{d}} {\mathcal {N}} (0,\sigma ^2 {\mathbf {V}}), \end{aligned}$$

we then obtain, as $n\rightarrow \infty$,

$$\begin{aligned} {\sqrt{n}}(\widehat{\varvec{\theta }}_{n}-{\varvec{\theta }}_{0}){\mathop \rightarrow \limits ^{d}}{\mathcal {N}} (0,\sigma ^2 \mathbf {V^{-1}}). \end{aligned}$$

Thus the proof is complete. $\square$