To reduce or not to reduce: a study on spatio-temporal surveillance

  • Junzhuo Chen
  • Chuljin Park
  • Seong-Hee KimEmail author
  • Yao Xie


The majority of control charts based on scan statistics for spatio-temporal surveillance use full observation vectors. In high-dimensional applications, dimension-reduction techniques are usually applied. Typically, the dimension reduction is conducted as a post-processing step rather than in the data acquisition stage and thus, a full sample covariance matrix is required. When the dimensionality of data is high, (i) the sample covariance matrix tends to be ill-conditioned due to a limited number of samples; (ii) the inversion of such a sample covariance matrix causes numerical issues; and (iii) aggregating information from all variables may lead to high communication costs in sensor networks. In this paper, we propose a set of reduced-dimension (RD) control charts that perform dimension reduction during the data acquisition process by spatial scanning. The proposed methods avoid computational difficulties and possibly high communication costs. We derive a theoretical measure that characterizes the performance difference between the RD approach and the full observation approach. The numerical results show that the RD approach has little performance loss under several commonly used spatial models while enjoying all the benefits of implementation. A case study on water quality monitoring demonstrates the effectiveness of the proposed methods in real applications.


Reliability Scan statistics Spatio-temporal surveillance Statistical process control (SPC) Statistical computing 



This material is based upon work supported by NSF under Grants CMMI-1538746.

Supplementary material

10651_2019_425_MOESM1_ESM.pdf (800 kb)
Supplementary material 1 (pdf 799 KB)


  1. Bartram J, Ballance R (1996) Water quality monitoring: a practical guide to the design and implementation of freshwater quality studies and monitoring programmes. CRC Press, New YorkGoogle Scholar
  2. Bodnar O, Schmid W (2005) Multivariate control charts based on a projection approach. Allgemeines Statistisches Archiv 89:75–93CrossRefGoogle Scholar
  3. Brynjarsdóttir J, Berliner LM (2014) Dimension-reduced modeling of spatio-temporal processes. J Am Stat Assoc 109:1647–1659CrossRefGoogle Scholar
  4. Corless RM, Gonnet GH, Hare DE, Jeffrey DJ, Knuth DE (1996) On the LambertW function. Adv Comput Math 5:329–359CrossRefGoogle Scholar
  5. Crosier RB (1988) Multivariate generalizations of cumulative sum quality-control schemes. Technometrics 30:291–303CrossRefGoogle Scholar
  6. Gaetan C, Guyon X (2010) Spatial statistics and modeling. Springer, New YorkCrossRefGoogle Scholar
  7. Guerriero M, Willett P, Glaz J (2009) Distributed target detection in sensor networks using scan statistics. In: IEEE transactions on signal processing, vol. 57, pp 2629–2639Google Scholar
  8. He Q, Zhou S (2014) Discriminant locality preserving projection chart for statistical monitoring of manufacturing processes. Int J Prod Res 52:5286–5300CrossRefGoogle Scholar
  9. Healy JD (1987) A note on multivariate CUSUM procedures. Technometrics 29:409–412CrossRefGoogle Scholar
  10. Hotelling, H (1947) “Multivariate quality control,” Techniques of statistical analysisGoogle Scholar
  11. Jiang W, Han SW, Tsui K-L, Woodall WH (2011) Spatiotemporal surveillance methods in the presence of spatial correlation. Stat Med 30:569–583CrossRefGoogle Scholar
  12. Kim S-H, Alexopoulos C, Tsui K-L, Wilson JR (2007) A distribution-free tabular CUSUM chart for autocorrelated data. IIE Trans 39:317–330CrossRefGoogle Scholar
  13. Lee J, Hur Y, Kim S-H, Wilson JR (2012) Monitoring nonlinear profiles using a wavelet-based distribution-free CUSUM chart. Int J Prod Res 50:6574–6594CrossRefGoogle Scholar
  14. Lee ML, Goldsman D, Kim S-H (2015) Robust distribution-free multivariate CUSUM charts for spatiotemporal biosurveillance in the presence of spatial correlation. IIE Trans Healthc Syst Eng 5:74–88CrossRefGoogle Scholar
  15. Lee ML, Goldsman D, Kim S-H, Tsui K-L (2014) Spatiotemporal biosurveillance with spatial clusters: control limit approximation and impact of spatial correlation. IIE Trans 46:813–827CrossRefGoogle Scholar
  16. Liu K, Mei Y, Shi J (2015) An adaptive sampling strategy for online high-dimensional process monitoring. Technometrics 57:305–319CrossRefGoogle Scholar
  17. Liu K, Zhang R, Mei Y (2016) Scalable SUM-Shrinkage Schemes for Distributed Monitoring Large-Scale Data Streams. arXiv:1603.08652
  18. Lowry CA, Woodall WH, Champ CW, Rigdon SE (1992) A multivariate exponentially weighted moving average control chart. Technometrics 34:46–53CrossRefGoogle Scholar
  19. Mishin D, Brantner-Magee K, Czako F, Szalay AS (2014) “Real time change point detection by incremental PCA in large scale sensor data,” In: High performance extreme computing conference (HPEC), 2014 IEEE. IEEE pp 1–6Google Scholar
  20. Page ES (1954) Continuous inspection schemes. Biometrika 41:100–115CrossRefGoogle Scholar
  21. Park Y, Baek SH, Kim S-H, Tsui K-L (2014) Statistical process control-based intrusion detection and monitoring. Qual Reliab Eng Int 30:257–273CrossRefGoogle Scholar
  22. Ripley BD (2005) Spatial statistics, vol 575. Wiley, New YorkGoogle Scholar
  23. Rogerson PA (1997) Surveillance systems for monitoring the development of spatial patterns. Stat Med 16:2081–2093CrossRefGoogle Scholar
  24. Rogerson PA, Yamada I (2004) Approaches to syndromic surveillance when data consist of small regional counts. Morbidity and Mortality Weekly Report pp 79–85Google Scholar
  25. Runger GC (1996) Projections and the U-squared multivariate control chart. J Qual Technol 28:313–319CrossRefGoogle Scholar
  26. Skubałlska-Rafajlowicz E (2013) Random projections and Hotelling’s T2 statistics for change detection in high-dimensional data analysis. Int J Appl Math Comput Sci 23:447–461CrossRefGoogle Scholar
  27. Sonesson C (2007) A CUSUM framework for detection of space–time disease clusters using scan statistics. Stat Med 26:4770–4789CrossRefGoogle Scholar
  28. Spiegelhalter D, Sherlaw-Johnson C, Bardsley M, Blunt I, Wood C, Grigg O (2012) Statistical methods for healthcare regulation: rating, screening and surveillance. J R Stat Soc 175:1–47CrossRefGoogle Scholar
  29. Tango T (1995) A class of tests for detecting ‘general’and ‘focused’clustering of rare diseases. Stat Med 14:2323–2334CrossRefGoogle Scholar
  30. Telci IT, Aral MM (2011) Contaminant source location identification in river networks using water quality monitoring systems for exposure analysis. Water Qual Expos Health 2:205–218CrossRefGoogle Scholar
  31. Tsui K-L, Chiu W, Gierlich P, Goldsman D, Liu X, Maschek T (2008) A review of healthcare, public health, and syndromic surveillance. Qual Eng 20:435–450CrossRefGoogle Scholar
  32. Ver Hoef JM, Peterson EE (2010) A moving average approach for spatial statistical models of stream networks. J Am Stat Assoc 105:6–18CrossRefGoogle Scholar
  33. Wang H, Kim S-H, Huo X, Hur Y, Wilson JR (2015a) Monitoring nonlinear profiles adaptively with a wavelet-based distribution-free CUSUM chart. Int J Prod Res 53:4648–4667CrossRefGoogle Scholar
  34. Wang K, Jiang W, Li B (2015b) A spatial variable selection method for monitoring product surface. Int J Prod Res 54:4161–4181CrossRefGoogle Scholar
  35. Xie Y, Huang J, Willett R (2013) Change-point detection for high-dimensional time series with missing data. IEEE J Select Top Sign Process. 7:12–27CrossRefGoogle Scholar
  36. Xie Y, Siegmund D (2013) Sequential multi-sensor change-point detection. Annal Stat 41:670–692CrossRefGoogle Scholar
  37. Xie, Y, Wang M, Thompson A (2015) Sketching for sequential change-point detection. In: 2015 IEEE global conference on signal and information processing (GlobalSIP), IEEE, pp 78–82Google Scholar
  38. Yamamoto M, Hwang H (2017) Dimension-reduced clustering of functional data via subspace separation. J Classif 34:294–326CrossRefGoogle Scholar
  39. Zhang F (2006) The Schur complement and its applications, vol 4. Springer, New YorkGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Junzhuo Chen
    • 1
  • Chuljin Park
    • 2
  • Seong-Hee Kim
    • 1
    Email author
  • Yao Xie
    • 1
  1. 1.H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of TechnologyAtlantaUSA
  2. 2.Department of Industrial EngineeringHanyang UniversitySeoulSouth Korea

Personalised recommendations