Markov Chains pp 141-176 | Cite as

Higher-Order Markov Chains

  • Wai-Ki Ching
  • Ximin Huang
  • Michael K. Ng
  • Tak-Kuen Siu
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 189)


Data sequences or time series occur frequently in many real world applications. One of the most important steps in analyzing a data sequence (or time series) is the selection of an appropriate mathematical model for the data. This is because it helps in predictions, hypothesis testing and rule discovery.


Markov Chain Linear Programming Problem Risky Asset Markov Chain Model Hide Markov Chain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Adke S, Deshmukh D (1988) Limit distribution of a high order Markov chain. J Roy Stat Soc Ser B 50:105–108Google Scholar
  2. 2.
    Albrecht D, Zukerman I, Nicholson A (1999) Pre-sending documents on the WWW: a comparative study. In: Proceedings of the sixteenth international joint conference on artificial intelligence IJCAI99Google Scholar
  3. 5.
    Artzner P, Delbaen F, Eber J, Heath D (1997) Thinking coherently. Risk 10:68–71Google Scholar
  4. 6.
    Artzner P, Delbaen F, Eber J, Heath D (1999) Coherent measures of risk. Math Finance 9:203–228CrossRefGoogle Scholar
  5. 17.
    Best P (1998) Implementing value at risk. Wiley, EnglandCrossRefGoogle Scholar
  6. 25.
    Brockwell P, Davis R (1991) Time series: theory and methods. Springer, New YorkCrossRefGoogle Scholar
  7. 29.
    Buffington J, Elliott R (2002) Regime switching and European options. In: Stochastic theory and control, proceedings of a workshop, Lawrence, K.S. Springer, Berlin, pp 73–81Google Scholar
  8. 30.
    Buffington J, Elliott R (2002) American options with regime switching. Int J Theoret Appl Finance 5:497–514CrossRefGoogle Scholar
  9. 51.
    Ching W, Fung E, Ng M (2003) A higher-order Markov model for the Newsboy’s problem. JOper Res Soc 54:291–298CrossRefGoogle Scholar
  10. 52.
    Ching W, Fung E, Ng M (2004) Higher-order Markov chain models for categorical data sequences. Int J Naval Res Logist 51:557–574CrossRefGoogle Scholar
  11. 53.
    Ching W, Fung E, Ng M (2004) Building higher-order Markov chain models with EXCEL. Int J Math Educ Sci Technol 35:921–932CrossRefGoogle Scholar
  12. 57.
    Ching W, Ng M (2003) Recent advance in data mining and modeling. World Scientific, SingaporeGoogle Scholar
  13. 68.
    Ching W, Ng M, Zhang S (2005) On computation with higher-order Markov chain. In: Zhang W, Chen Z, Glowinski R, Tong W (eds) Current trends in high performance computing and its applications proceedings of the international conference on high performance computing and applications, 8–10 August 2004. Springer, Shanghai, China, pp 15–24Google Scholar
  14. 79.
    Davis P (1979) Circulant matrices. Wiley, New YorkGoogle Scholar
  15. 81.
    Duffie D, Pan J (1997) An overview of value at risk. J Derivat 4(3):7–49CrossRefGoogle Scholar
  16. 83.
    Elliott R (1993) New finite-dimensional filters and smoothers for noisily observed Markov chains. IEEE Trans Inform Theory 39(1):265–271CrossRefGoogle Scholar
  17. 84.
    Elliott R, Aggoun L, Moore J (1994) Hidden Markov models: estimation and control. Springer, New YorkGoogle Scholar
  18. 85.
    Elliott R, Chan L, Siu T (2005) Option pricing and Esscher transform under regime switching. Ann Finance 1(4):423–432CrossRefGoogle Scholar
  19. 86.
    Elliott R, van der Hoek J (1997) An application of hidden Markov models to asset allocation problems. Finance Stochast 3:229–238CrossRefGoogle Scholar
  20. 87.
    Elliott R, Hunter W, Jamieson B (1998) Drift and volatility estimation in discrete time. J Econ Dyn Contr 22:209–218CrossRefGoogle Scholar
  21. 88.
    Elliott R, Liew C, Siu T (2011) On filtering and estimation of a threshold stochastic volatility model. Appl Math Comput 218(1):61–75CrossRefGoogle Scholar
  22. 89.
    Elliott R, Malcolm W, Tsoi A (2003) Robust parameter estimation for asset price models with Markov modulated volatilities. J Econ Dyn Contr 27(8):1391–1409CrossRefGoogle Scholar
  23. 90.
    Elliott R, Mamon R (2003) An interest rate model with a Markovian mean-reverting level. Quant Finance 2:454–458Google Scholar
  24. 91.
    Elliott R, Miao H (2006) Stochastic volatility model with filtering. Stochast Anal Appl 24:661–683CrossRefGoogle Scholar
  25. 92.
    Elliott R, Miao H (2009) VaR and expected shortfall: a non-normal regime switching framework. Quant Finance 9:747–755CrossRefGoogle Scholar
  26. 93.
    Elliott R, Siu T (2009) On Markov-modulated exponential-affine bond price formulae. Appl Math Finance 16:1–15CrossRefGoogle Scholar
  27. 94.
    Elliott R, Siu T (2010) On risk minimizing portfolios under a Markovian regime-switching black-scholes economy. Ann Oper Res 176:271–291CrossRefGoogle Scholar
  28. 95.
    Elliott R, Siu T (2011) Pricing and hedging contingent claims with regime switching risk. Commun Math Sci 9:477–498Google Scholar
  29. 96.
    Elliott R, Siu T (2011) A stochastic differential game for optimal investment of an insurer with regime switching. Quant Finance 11:365–380CrossRefGoogle Scholar
  30. 97.
    Elliott R, Siu T, Fung E (2011) Filtering a nonlinear stochastic volatility model. Nonlinear Dynam 67(2):1295–1313CrossRefGoogle Scholar
  31. 98.
    Fang S, Puthenpura S (1993) Linear optimization and extensions. Prentice-Hall, New JerseyGoogle Scholar
  32. 107.
    Goldfeld S, Quandt R (1973) A Markov model for switching regressions. J Econ 1:3–16Google Scholar
  33. 109.
    Gowda K, Diday E (1991) Symbolic clustering using a new dissimilarity measure. Pattern Recogn 24(6):567–578CrossRefGoogle Scholar
  34. 110.
    Guo X (2001) Information and option pricing. Quant Finance 1:38–44CrossRefGoogle Scholar
  35. 112.
    Hamilton J (1989) A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica 57:357–384CrossRefGoogle Scholar
  36. 120.
    Huang J, Ng M, Ching W, Cheung D, Ng J (2001) A cube model for web access sessions and cluster analysis. In: Kohavi R, Masand B, Spiliopoulou M and Srivastava J (eds) WEBKDD 2001, Workshop on mining web log data across all customer touch points. The seventh ACM SIGKDD international conference on knowledge discovery and data mining. Lecture notes in computer science. Springer, Berlin, pp 47–58Google Scholar
  37. 126.
    Joachims T, Freitag D, Mitchell T (1997) WebWatch: a tour guide for the world wide web. In: Proceedings of the fifteenth international joint conference on artificial intelligence IJCAI 97, pp 770–775Google Scholar
  38. 127.
    Jorion P (1997) Value at risk: the new benchmark for controlling market risk. McGraw-Hill, New YorkGoogle Scholar
  39. 146.
    Li W, Kwok M (1989) Some results on the estimation of a higher order Markov chain. Department of Statistics, The University of Hong KongGoogle Scholar
  40. 147.
    Lieberman H (1995) Letizia: an agent that assists web browsing. In: Proceedings of the fourteenth international joint conference on artificial intelligence IJCAI 95, pp 924–929Google Scholar
  41. 151.
    Logan J (1981) A structural model of the higher-order Markov process incorporating reversion effects. J Math Sociol 8: 75–89CrossRefGoogle Scholar
  42. 153.
    Luo S, Tsoi A (2007) Filtering of hidden weak Markov chain: discrete range obervations. In: Mamon R, Elliott R (eds) Hidden Markov models in finance. Springer, New York, pp 101–119Google Scholar
  43. 155.
    MacDonald I, Zucchini W (1997) Hidden Markov and other models for discrete-valued time series. Chapman & Hall, LondonGoogle Scholar
  44. 156.
    Mamon R, Elliott R (2007) Hidden Markov models in finance. Springer’s international series in operations research and management science, vol 104. Springer, New YorkGoogle Scholar
  45. 157.
    McNeil A, Frey R, Embrechts P (2005) Quantitative risk management: concepts, techniques and tools. Princeton University Press, PrincetonGoogle Scholar
  46. 165.
    Nahmias S (1981) Managing repairable item inventory systems: a review in TIMS studies. Manag Sci 16:253–277Google Scholar
  47. 166.
    Naik V (1993) Option valuation and hedging strategies with jumps in the volatility of asset returns. J Finance 48:1969–1984CrossRefGoogle Scholar
  48. 171.
    Pliska S (2003) Introduction to mathematical finance: discrete time models. Blackwell, OxfordGoogle Scholar
  49. 174.
    Quandt R (1958) The estimation of parameters of linear regression system obeying two separate regimes. J Am Stat Assoc 55:873–880CrossRefGoogle Scholar
  50. 176.
    Raftery A (1985) A model for high-order Markov chains. J Roy Stat Soc Ser B 47:528–539Google Scholar
  51. 177.
    Raftery A, Tavare S (1994) Estimation and modelling repeated patterns in high order Markov chains with the mixture transition distribution model. J Appl Stat 43:179–199CrossRefGoogle Scholar
  52. 183.
    Shahabi C, Faisal A, Kashani F, Faruque J (2000) INSITE: a tool for real time knowledge discovery from users web navigation. In: Proceedings of VLDB2000, Cairo, EgyptGoogle Scholar
  53. 184.
    Siu T (2008) A game theoretic approach to option valuation under Markovian regime-switching models. Insur Math Econ 42:1146–1158CrossRefGoogle Scholar
  54. 185.
    Siu T (2010) A Markov regime switching marked point process for short rate analysis with credit risk. Int J Stochast Anal 2010, Article ID 870516, 18 pagesGoogle Scholar
  55. 186.
    Siu T (2010) Bond pricing under a Markovian regime-switching jump-augmented vasicek model via stochastic flows. Appl Math Comput 216:3184–3190CrossRefGoogle Scholar
  56. 189.
    Siu T, Ching E, Fung E, Ng M, Li X (2009) A higher-order Markov-switching model for risk measurement. Comput Math Appl 58:1–10CrossRefGoogle Scholar
  57. 197.
    Tong H (1978) On a threshold model. In: Pattern recognition and signal processing. Sijthoff and Noordhoff, The NetherlandsGoogle Scholar
  58. 198.
    Tong H (1983) Threshold models in non-linear time series analysis. Springer, BerlinCrossRefGoogle Scholar
  59. 199.
    Tong H (1990) Non-linear time series: a dynamical system approach. Oxford University Press, OxfordGoogle Scholar
  60. 200.
    Tong H, Lim K (1980) Threshold autoregression, limit cycles and cyclical data. J Roy Stat Soc Ser B, Methodol 42:245–292Google Scholar
  61. 201.
    Tsoi A (2007) Discrete time weak Markov term structure model (preprint)Google Scholar
  62. 202.
    Tsoi A (2007) Discrete time reversal and duality of weak Markov chain (preprint)Google Scholar
  63. 205.
    Wang Y (1992) Approximation k th-order two-state Markov chains. J Appl Probab 29:861–868CrossRefGoogle Scholar
  64. 210.
    Yang Q, Huang Z, Ng M (2003) A data cube model for prediction-based web prefetching. J Intell Inform Syst 20:11–30CrossRefGoogle Scholar
  65. 211.
    Yin G, Zhou X (2004) Markowitz’s mean-variance portfolio selection with regime switching: from discrete-time models to their continuous-time limits. IEEE Trans Automat Contr 49:349–360CrossRefGoogle Scholar
  66. 215.
    Zhou X, Yin G (2003) Markowitz’s mean-variance portfolio selection with regime switching: a continuous time model. SIAM J Contr Optim 42:1466–1482CrossRefGoogle Scholar
  67. 216.
    Zhu D, Ching W (2011) A note on the stationary property of high-dimensional Markov chain models. Int J Pure Appl Math 66:321–330Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Wai-Ki Ching
    • 1
  • Ximin Huang
    • 2
  • Michael K. Ng
    • 3
  • Tak-Kuen Siu
    • 4
  1. 1.Department of MathematicsThe University of Hong KongHong KongHong Kong, SAR
  2. 2.College of ManagementGeorgia Institute of TechnologyAtlantaUSA
  3. 3.Department of MathematicsHong Kong Baptist UniversityKowloon TongHong Kong SAR
  4. 4.Cass Business SchoolCity University LondonLondonUK

Personalised recommendations