Markov Chain Model for Linear Systems
This chapter contributes to the mean square and almost sure convergence analysis of ILC algorithms for linear systems with data dropouts. The data dropout model considered in this chapter is described by a Markov chain model. The proposed stochastic approximation-type ILC algorithm copes with unreliable communication conditions including stochastic measurement noises, random transmission gains, and Markov data dropouts. Under mild assumptions, we establish the mean square and almost sure convergence of the proposed algorithm for both conventional and general Markov chain models, using time-invariant and varying step sizes.