Discrete-Time Markovian Jump Linear Systems

Abstract

The research is oriented to the control of discrete — time linear systems with randomly changing parameters which can be described by a finite — state Markov chain. The cost critrion is a quadratic form of the controls and states of the system. The criterion parameters also depend on the states of the Markov chain. Two models of observation of the Markov chain are adopted — delay for one step and undelay. It is shown that under appropriate mean square detectability and stability conditions the infinite horizon optimal control problem for the general case of Markovian jump linear quadratic systems has a unique solution when the control system is mean square stable. Necessary and sufficient conditions are given to determine if a system is mean square stable.