Methodology and Computing in Applied Probability

, Volume 21, Issue 4, pp 1407–1429 | Cite as

Reliability and Survival Analysis for Drifting Markov Models: Modeling and Estimation

  • Vlad Stefan BarbuEmail author
  • Nicolas Vergne


In this work we focus on multi-state systems modeled by means of a particular class of non-homogeneous Markov processes introduced in Vergne (Stat Appl Genet Mol Biol 7(1):1–45, 2008), called drifting Markov processes. The main idea behind this type of processes is to consider a non-homogeneity that is “smooth”, of a known shape. More precisely, the Markov transition matrix is assumed to be a linear (polynomial) function of two (several) Markov transition matrices. For this class of systems, we first obtain explicit expressions for reliability/survival indicators of drifting Markov models, like reliability, availability, maintainability and failure rates. Then, under different statistical settings, we estimate the parameters of the model, obtain plug-in estimators of the associated reliability/survival indicators and investigate the consistency of the estimators. The quality of the proposed estimators and the model validation is illustrated through numerical experiments.


Drifting Markov chains Multi-state systems Reliability theory Survival analysis Estimation Asymptotic properties 

Mathematics Subject Classification (2010)

60J10 60K15 90B25 62N02 62F12 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



This research work was partially supported by the projects MMATFAS - Mathematical Methods for System Reliability and Survival Analysis (2014–2015) and MOUSTIC–Random Models and Statistical, Informatics and Combinatorics Tools (2016–2019), within the Large Scale Research Networks from the Region of Normandy, France.

The authors would like to thank their colleagues Dr. Anatoly Lisnianski and Dr. Gregory Levitin from the Israel Electric Corporation Ltd. for pointing out important aspects and references on multi-state systems and also to Prof. Ilia Frenkel and his colleagues from Sami Shamoon College of Engineering, Israel, for organizing the series of conferences International Symposium on Stochastic Models in Reliability Engineering, Life Science and Operations Management, to which, indirectly, the research developed in this article owes much.


  1. Alvarez EE (2005) Estimation in stationary Markov renewal processes, with application to earthquake forecasting in Turkey. Methodol Comput Appl Probab 7(1):119–130. MathSciNetCrossRefzbMATHGoogle Scholar
  2. Barbu VS, Limnios N (2004) Discrete time semi-Markov processes for reliability and survival analysis–a nonparametric estimation approach. In: Balakrishnan N, Nikulin M, Mesbah M, Limnios N (eds) Parametric and semiparametric models with applications to reliability, survival analysis and quality of life, Collection Statistics for Industry and Technology. Birkhäuser, Boston, pp 487–502Google Scholar
  3. Barbu VS, Limnios N (2006) Nonparametric estimation for failure rate functions of discrete time semi-Markov processes. In: Nikulin M, Commenges D, Huber-Carol C (eds) Probability, statistics and modelling in public health. Springer, pp 53–72Google Scholar
  4. Barbu VS, Limnios N (2008a) Reliability of semi-Markov systems in discrete time: modeling and estimation. In: Misra KB (ed) Handbook on performability engineering. Springer, pp 369–380Google Scholar
  5. Barbu V S, Limnios N (2008b) Semi-markov chains and hidden semi-Markov models toward applications-their use in reliability and DNA analysis. Lecture Notes in Statistics, vol 191. Springer, New YorkzbMATHGoogle Scholar
  6. Barbu VS, Limnios N (2010) Some algebraic methods in semi-Markov processes. In: Viana MAG, Wynn HP (eds) Algebraic methods in statistics and probability, vol 2, series contemporary mathematics edited by AMS. Urbana, pp 19-35Google Scholar
  7. Barbu VS, Karagrigoriou A, Makrides A (2016) Semi-markov modelling for multi-state systems. Methodol Comput Appl Probab 19:1011–1028. MathSciNetCrossRefzbMATHGoogle Scholar
  8. Barlow RE, Marshall AW, Prochan F (1963) Properties of probability distributions with monotone hazard rate. Ann Math Statist 34:375–389MathSciNetCrossRefGoogle Scholar
  9. Cappé O, Moulines E, Ryden T (2005) Inference in hidden Markov models. Springer, New YorkCrossRefGoogle Scholar
  10. Chiquet J, Limnios N (2006) Estimating stochastic dynamical systems driven by a continuous-time jump Markov process. Methodol Comput Appl Probab 8(4):431–447. MathSciNetCrossRefzbMATHGoogle Scholar
  11. Chryssaphinou O, Limnios N, Malefaki S (2011) Multi-state reliability systems under discrete time semi-Markovian hypothesis. IEEE Trans Reliab 60(1):80–87. CrossRefGoogle Scholar
  12. D’Amico G, Gismondi F, Janssen J, Manca R (2015a) Discrete time homogeneous Markov processes for the study of the basic risk processes. Methodol Comput Appl Probab 17(4):983–998. MathSciNetCrossRefzbMATHGoogle Scholar
  13. D’Amico G, Petroni F, Prattico F (2015b) Performance analysis of second order semi-Markov chains: an application to wind energy production. Methodol Comput Appl Probab 17(3):781–794. MathSciNetCrossRefzbMATHGoogle Scholar
  14. Hou Y, Limnios N, Schön W (2017) On the existence and uniqueness of solution of MRE and applications. Methodology and Computing in Applied Probability. MathSciNetCrossRefGoogle Scholar
  15. Koutras VP, Malefaki S, Platis AN (2017) Optimization of the dependability and performance measures of a generic model for multi-state deteriorating systems under maintenance. Reliab Eng Syst Saf 166:73–86CrossRefGoogle Scholar
  16. Levitin G (2005) The universal generating function in reliability analysis and optimization. Springer, LondonGoogle Scholar
  17. Limnios N (2012) Reliability measures of semi-Markov systems with general state space. Methodol Comput Appl Probab 14(4):895–917. MathSciNetCrossRefzbMATHGoogle Scholar
  18. Limnios N, Oprişan G (2001) Semi-Markov processes and reliability. Birkhäuser, BostonCrossRefGoogle Scholar
  19. Limnios N, Ouhbi B (2003) Empirical estimators of reliability and related functions for semi-Markov systems. In: Lindqvist BH, Doksum KA (eds) Mathematical and statistical methods in reliability. World Scientific, Singapore, vol, 7, pp 469-484CrossRefGoogle Scholar
  20. Limnios N, Ouhbi B (2006) Nonparametric estimation of some important indicators in reliability for semi-Markov processes. Stat Methodol 3:341–350MathSciNetCrossRefGoogle Scholar
  21. Lisnianski A (2001) Estimation of boundary points for continuum-state system reliability measures. Reliab Eng Syst Saf 74:81–88CrossRefGoogle Scholar
  22. Lisnianski A, Levitin G (2003) Multi-state system reliability: assessment optimization and applications. World Scientific, SingaporeCrossRefGoogle Scholar
  23. Lisnianski A, Frenkel I, Ding Y (2010) Multi-state system reliability analysis and optimization for engineers and industrial managers. Springer, LondonCrossRefGoogle Scholar
  24. Lisnianski A, Elmakias D, Laredo D, Ben Haim H (2012) A multi-state Markov model for a short-term reliability analysis of a power generating unit. Reliab Eng Syst Saf 98(1):1–6CrossRefGoogle Scholar
  25. MacDonald IL, Zucchini W (1997) Hidden Markov and other models for discrete-valued time series. Chapman and Hall, LondonzbMATHGoogle Scholar
  26. Mathieu E, Foucher Y, Dellamonica P, Daures JP (2007) Parametric and non homogeneous semi-Markov process for HIV control. Methodol Comput Appl Probab 9(3):389–397. MathSciNetCrossRefzbMATHGoogle Scholar
  27. Natvig B (2011) Multistate systems reliability. Theory with applications. Wiley, New YorkCrossRefGoogle Scholar
  28. Ouhbi B, Limnios N (1999) Non-parametric estimation for semi-Markov processes based on its hazard rate functions. Stat Infer Stoch Process 2(2):151–173CrossRefGoogle Scholar
  29. Roy D, Gupta R (1992) Classification of discrete lives. Microelectron Reliab 32(10):1459–1473CrossRefGoogle Scholar
  30. Ruiz-Castro JE (2016) Markov counting and reward processes for analysing the performance of a complex system subject to random inspections. Reliab Eng Syst Saf 145:155–168CrossRefGoogle Scholar
  31. Sadek A, Limnios N (2002) Asymptotic properties for maximum likelihood estimators for reliability and failure rates of Markov chains. Comm Statist Theory Methods 31(10):1837–1861MathSciNetCrossRefGoogle Scholar
  32. Sadek A, Limnios N (2005) Nonparametric estimation of reliability and survival function for continuous-time finite Markov processes. J Stat Plann Infer 133(1):1–21MathSciNetCrossRefGoogle Scholar
  33. Silvestrov D, Manca R (2017) Reward algorithms for semi-Markov processes. Methodology and Computing in Applied Probability. MathSciNetCrossRefGoogle Scholar
  34. Vasiliadis G, Tsaklidis G (2008) On the distributions of the state sizes of closed continuous time homogeneous Markov systems. Methodol Comput Appl Probab 11(4):561–582. MathSciNetCrossRefzbMATHGoogle Scholar
  35. Vassiliou P-CG, Moysiadis TP (2010) G-inhomogeneous Markov systems of high order. Methodol Comput Appl Probab 12(2):271–292. MathSciNetCrossRefzbMATHGoogle Scholar
  36. Vergne N (2008) Drifting Markov models with polynomial drift and applications to DNA sequences. Stat Appl Genet Mol Biol 7(1):1–45MathSciNetCrossRefGoogle Scholar
  37. Votsi I, Limnios N, Tsaklidis G, Papadimitriou E (2012) Estimation of the expected number of earthquake occurrences based on semi-Markov models. Methodol Comput Appl Probab 14(3):685–703. MathSciNetCrossRefzbMATHGoogle Scholar
  38. Xie M, Gaudoin O, Bracquemond C (2002) Redefining failure rate function for discrete distributions. Int J Reliab Qual Saf Eng 9(3):275–285CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Laboratoire de Mathématiques Raphaël SalemUniversité de RouenSaint-Étienne-du-RouvrayFrance

Personalised recommendations