Change-Point Detection Method for Clinical Decision Support System Rule Monitoring

  • Siqi LiuEmail author
  • Adam Wright
  • Milos Hauskrecht
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10259)


A clinical decision support system (CDSS) and its components can malfunction due to various reasons. Monitoring the system and detecting its malfunctions can help one to avoid any potential mistakes and associated costs. In this paper, we investigate the problem of detecting changes in the CDSS operation, in particular its monitoring and alerting subsystem, by monitoring its rule firing counts. The detection should be performed online, that is whenever a new datum arrives, we want to have a score indicating how likely there is a change in the system. We develop a new method based on Seasonal-Trend decomposition and likelihood ratio statistics to detect the changes. Experiments on real and simulated data show that our method has a lower delay in detection compared with existing change-point detection methods.


Electronic Health Record Clinical Decision Support System Likelihood Ratio Statistic Online Detection Rule Firing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This research was supported by grants R01-LM011966 and R01-GM088224 from the NIH. The content of this paper is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.


  1. 1.
    Bartlett, M.S.: The use of transformations. Biometrics 3(1), 39–52 (1947)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Chen, C., Liu, L.M.: Joint estimation of model parameters and outlier effects in time series. J. Am. Stat. Assoc. 88(421), 284–297 (1993)zbMATHGoogle Scholar
  3. 3.
    Chen, J., Gupta, A.K.: Parametric Statistical Change Point Analysis. Birkhäuser Boston, Boston (2012)CrossRefzbMATHGoogle Scholar
  4. 4.
    Cleveland, R.B., Cleveland, W.S., McRae, J.E., Terpenning, I.: STL: a seasonal-trend decomposition procedure based on loess. J. Off. Stat. 6(1), 3–73 (1990)Google Scholar
  5. 5.
    Cleveland, W.S., Cleveland, W.S.: Robust locally weighted regression and smoothing scatterplots. J. Am. Stat. Assoc. 74(368), 829–836 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Cleveland, W.S., Devlin, S.J.: Locally weighted regression: an approach to regression analysis by local fitting. J. Am. Stat. Assoc. 83(403), 596–610 (1988)CrossRefzbMATHGoogle Scholar
  7. 7.
    Fawcett, T., Provost, F.: Activity monitoring: noticing interesting changes in behavior. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, vol. 1, pp. 53–62 (1999)Google Scholar
  8. 8.
    Fox, A.J.: Outliers in time series. J. Roy. Stat. Soc.: Ser. B (Methodol.) 34(3), 350–363 (1972)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Fryzlewicz, P.: Wild binary segmentation for multiple change-point detection. Ann. Stat. 42(6), 2243–2281 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Garg, A.X., Adhikari, N.K.J., McDonald, H., Rosas-Arellano, M.P., Devereaux, P.J., Beyene, J., Sam, J., Haynes, R.B.: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 293(10), 1223–1238 (2005)CrossRefGoogle Scholar
  11. 11.
    Hauskrecht, M., Batal, I., Hong, C., Nguyen, Q., Cooper, G.F., Visweswaran, S., Clermont, G.: Outlier-based detection of unusual patient-management actions: an ICU study. J. Biomed. Inform. 64, 211–221 (2016)CrossRefGoogle Scholar
  12. 12.
    Kawahara, Y., Sugiyama, M.: Change-point detection in time-series data by direct density-ratio estimation. In: SIAM International Conference on Data Mining, pp. 389–400. Society for Industrial and Applied Mathematics, April 2009Google Scholar
  13. 13.
    Killick, R., Eckley, I.: changepoint: an R package for changepoint analysis. J. Stat. Softw. 58(3), 1–19 (2014)CrossRefGoogle Scholar
  14. 14.
    Killick, R., Fearnhead, P., Eckley, I.A.: Optimal detection of changepoints with a linear computational cost. J. Am. Stat. Assoc. 107(500), 1590–1598 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Liu, C., Rubin, D.B.: ML estimation of the t distribution using EM and its extensions. ECM ECME. Stat. Sin. 5, 19–39 (1995)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Liu, S., Wright, A., Hauskrecht, M.: Online conditional outlier detection in nonstationary time series. In: FLAIRS Conference (2017)Google Scholar
  17. 17.
    Page, E.S.: Continuous inspection schemes. Biometrika 41(1/2), 100–115 (1954)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Pettitt, A.N.: A non-parametric approach to the change-point problem. J. Roy. Stat. Soc.: Ser. C (Appl. Stat.) 28(2), 126–135 (1979)MathSciNetzbMATHGoogle Scholar
  19. 19.
    R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2016)Google Scholar
  20. 20.
    Sen, A., Srivastava, M.S.: On tests for detecting change in mean. Ann. Stat. 3(1), 98–108 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Tsay, R.S.: Outliers, level shifts, and variance changes in time series. J. Forecast. 7(May 1987), 1–20 (1988)CrossRefGoogle Scholar
  22. 22.
    Wright, A., Hickman, T.T.T., McEvoy, D., Aaron, S., Ai, A., Andersen, J.M., Hussain, S., Ramoni, R., Fiskio, J., Sittig, D.F., Bates, D.W.: Analysis of clinical decision support system malfunctions: a case series and survey. J. Am. Med. Inform. Assoc. 23(6), 1068–1076 (2016)CrossRefGoogle Scholar
  23. 23.
    Yamanishi, K., Takeuchi, J.: A unifying framework for detecting outliers and change points from non-stationary time series data. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 676–681. ACM (2002)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of PittsburghPittsburghUSA
  2. 2.Brigham and Women’s Hospital and Harvard Medical SchoolBostonUSA

Personalised recommendations