Advertisement

Data Mining and Knowledge Discovery

, Volume 32, Issue 4, pp 1056–1073 | Cite as

Anomaly detection in spatiotemporal data via regularized non-negative tensor analysis

  • Chaoguang Lin
  • Qiuhan Zhu
  • Shunan Guo
  • Zhuochen Jin
  • Yu-Ru Lin
  • Nan Cao
Article

Abstract

Anomaly detection in multidimensional data is a challenging task. Detecting anomalous mobility patterns in a city needs to take spatial, temporal, and traffic information into consideration. Although existing techniques are able to extract spatiotemporal features for anomaly analysis, few systematic analysis about how different factors contribute to or affect the anomalous patterns has been proposed. In this paper, we propose a novel technique to localize spatiotemporal anomalous events based on tensor decomposition. The proposed method employs a spatial-feature-temporal tensor model and analyzes latent mobility patterns through unsupervised learning. We first train the model based on historical data and then use the model to capture the anomalies, i.e., the mobility patterns that are significantly different from the normal patterns. The proposed technique is evaluated based on the yellow-cab dataset collected from New York City. The results show several interesting latent mobility patterns and traffic anomalies that can be deemed as anomalous events in the city, suggesting the effectiveness of the proposed anomaly detection method.

Keywords

Tensor analysis Anomaly detection Outlier detection Urban computing Traffic analysis 

References

  1. Bahadori MT, Yu QR, Liu Y (2014) Fast multivariate spatio-temporal analysis via low rank tensor learning. In: Advances in neural information processing systems, pp 3491–3499Google Scholar
  2. Bai Y, Tezcan J, Cheng Q, Cheng J (2013) A multiway model for predicting earthquake ground motion. In: ACIS international conference on software engineering, artificial intelligence, networking and parallel/distributed computing (SNPD), pp 219–224Google Scholar
  3. Breunig MM, Kriegel HP, Ng RT, Sander J (2000) Lof: identifying density-based local outliers. ACM Sigmod Rec 29:93–104CrossRefGoogle Scholar
  4. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv (CSUR) 41(3):15CrossRefGoogle Scholar
  5. Chen Y, Zhou XS, Huang TS (2001) One-class svm for learning in image retrieval. IEEE Image Process 1:34–37Google Scholar
  6. Cichocki A, Zdunek R, Phan AH, Amari SI (2009) Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation. Wiley, New YorkCrossRefGoogle Scholar
  7. Fanaee-T H, Gama J (2015) Eigenevent: an algorithm for event detection from complex data streams in syndromic surveillance. Intell Data Anal 19(3):597–616CrossRefGoogle Scholar
  8. Fanaee-T H, Gama J (2016a) Tensor-based anomaly detection: an interdisciplinary survey. Knowl Based Syst 98:130–147CrossRefGoogle Scholar
  9. Fanaee-T H, Gama J (2016b) Event detection from traffic tensors: a hybrid model. Neurocomputing 203:22–33CrossRefGoogle Scholar
  10. Fanaee-T H, Gama J (2014) An eigenvector-based hotspot detection. arXiv preprint arXiv:1406.3191
  11. Gauvin L, Panisson A, Cattuto C (2014) Detecting the community structure and activity patterns of temporal networks: a non-negative tensor factorization approach. PloS ONE 9(1):e86028CrossRefGoogle Scholar
  12. Jiang M, Cui P, Faloutsos C (2016) Suspicious behavior detection: current trends and future directions. IEEE Intell Syst 31(1):31–39CrossRefGoogle Scholar
  13. Kim J, He Y, Park H (2014) Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework. J Global Optim 58(2):285–319MathSciNetCrossRefMATHGoogle Scholar
  14. Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500MathSciNetCrossRefMATHGoogle Scholar
  15. Kotsia I, Guo W, Patras I (2012) Higher rank support tensor machines for visual recognition. Pattern Recogn 45(12):4192–4203CrossRefMATHGoogle Scholar
  16. Liu S, Cui W, Wu Y, Liu M (2014) A survey on information visualization: recent advances and challenges. Visual Comput 30(12):1373–1393CrossRefGoogle Scholar
  17. Liu D, Weng D, Li Y, Bao J, Zheng Y, Qu H, Wu Y (2017) SmartAdP: Visual analytics of large-scale taxi trajectories for selecting billboard locations. IEEE Trans. Vis. Comput. Graphics 23(1):1–10CrossRefGoogle Scholar
  18. Liu Y, Zhou B, Chen F, Cheung DW (2016) Graph topic scan statistic for spatial event detection. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM, pp 489–498Google Scholar
  19. Mao HH, Wu CJ, Papalexakis EE, Faloutsos C, Lee KC, Kao TC (2014) Malspot: Multi2 malicious network behavior patterns analysis. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, pp 1–14Google Scholar
  20. Matsubara Y, Sakurai Y, Faloutsos C, Iwata T, Yoshikawa M (2012) Fast mining and forecasting of complex time-stamped events. In: Proceedings of the ACM SIGKDD international conference on Knowledge discovery and data mining, pp 271–279Google Scholar
  21. Nomikos P, MacGregor JF (1994) Monitoring batch processes using multiway principal component analysis. AIChE J 40(8):1361–1375CrossRefGoogle Scholar
  22. Papalexakis EE, Faloutsos C, Sidiropoulos ND (2012) Parcube: sparse parallelizable tensor decompositions. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 521–536Google Scholar
  23. Papalexakis E, Pelechrinis K, Faloutsos C (2014) Spotting misbehaviors in location-based social networks using tensors. In: Proceedings of the international conference on world wide web. ACM, pp 551–552Google Scholar
  24. Paschalidis IC, Smaragdakis G (2009) Spatio-temporal network anomaly detection by assessing deviations of empirical measures. IEEE/ACM Trans Netw (TON) 17(3):685–697CrossRefGoogle Scholar
  25. Prada MA, Dominguez M, Barrientos P, Garcia S (2012a) Dimensionality reduction for damage detection in engineering structures. Int J Mod Phys B 26(25):1246004CrossRefGoogle Scholar
  26. Prada MA, Toivola J, Kullaa J, HollméN J (2012b) Three-way analysis of structural health monitoring data. Neurocomputing 80:119–128CrossRefGoogle Scholar
  27. Rendle S (2012) Factorization machines with libfm. ACM Trans Intell Syst Technol (TIST) 3(3):57Google Scholar
  28. Rozenshtein P, Anagnostopoulos A, Gionis A, Tatti N (2014) Event detection in activity networks. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 1176–1185Google Scholar
  29. Shi L, Gangopadhyay A, Janeja VP (2015) Stensr: spatio-temporal tensor streams for anomaly detection and pattern discovery. Knowl Inf Syst 43(2):333CrossRefGoogle Scholar
  30. Sun GD, Liang R, Qu H, Wu Y (2017a) Embedding spatiotemporal information into maps by route-zooming. IEEE Trans. Vis. Comput. Graphics 23(5):1506–1519CrossRefGoogle Scholar
  31. Sun G, Tang T, Peng TQ, Liang R, Wu Y (2017b) Socialwave: visual analysis of spatio-temporal diffusion of information on social media. ACM Trans Intell Syst Technol 9(2):15CrossRefGoogle Scholar
  32. Sun J, Tao D, Papadimitriou S, Yu PS, Faloutsos C (2008) Incremental tensor analysis: Theory and applications. ACM Trans Knowl Discov Data (TKDD) 2(3):11Google Scholar
  33. Sun J, Tao D, Faloutsos C (2006) Beyond streams and graphs: dynamic tensor analysis. In: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 374–383Google Scholar
  34. Sun G, Wu YC, Liang RH, Liu SX (2013) A survey of visual analytics techniques and applications: state-of-the-art research and future challenges. J Comput Sci Tech 28(5):852–867CrossRefGoogle Scholar
  35. Tao D, Li X, Hu W, Maybank S, Wu X (2005) Supervised tensor learning. In: IEEE international conference on data miningGoogle Scholar
  36. Thai-Nghe N, Horváth T, Schmidt-Thieme L (2010) Factorization models for forecasting student performance. In: Educational Data Mining 2011Google Scholar
  37. Tian X, Zhang X, Deng X, Chen S (2009) Multiway kernel independent component analysis based on feature samples for batch process monitoring. Neurocomputing 72(7):1584–1596CrossRefGoogle Scholar
  38. Tork HF, Oliveira M, Gama J, Malinowski S, Morla R (2012) Event and anomaly detection using tucker3 decomposition. In: Workshop on ubiquitous data mining, p 8Google Scholar
  39. Wang XR, Lizier JT, Obst O, Prokopenko M, Wang P (2008) Spatiotemporal anomaly detection in gas monitoring sensor networks. In: Wireless sensor networks: 5th European conference, EWSN 2008. Springer, pp 90–105Google Scholar
  40. Wang J, Gao F, Cui P, Li C, Xiong Z (2014) Discovering urban spatio-temporal structure from time-evolving traffic networks. In: Asia-Pacific web conference. Springer, pp 93–104Google Scholar
  41. Weng D, Zhu H, Bao J, Zheng Y, Wu Y (2018) Homefinder revisited: finding ideal homes with reachability centric multi-criteria decision making. In Proceedings of ACM CHIGoogle Scholar
  42. Wu Y, Lan J, Shu X, Ji C, Zhao K, Wang J, Zhang H (2018) ITTVIS: Interactive visualization of table tennis data. IEEE Trans Visualization and Comp Graphics 24(1):709–718CrossRefGoogle Scholar
  43. Xia J, Chen W, Hou Y, Hu W, Huang X, Ebertk DS (2016) DimScanner: A relation-based visual exploration approach towards data dimension inspection. In: IEEE conference on visual analytics science and technology (VAST). pp 81–90Google Scholar
  44. Xie C, Chen W, Huang X, Hu Y, Barlowe S, Yang J (2014) VAET: A visual analytics approach for e-transactions time-series. IEEE Trans. Vis. Comput. Graphics 20(12):1743–1752CrossRefGoogle Scholar
  45. Xu P, Mei H, Ren L, Chen W (2017) ViDX: Visual diagnostics of assembly line performance in smart factories. IEEE Trans. Vis. Comput. Graphics 23(1):291–300CrossRefGoogle Scholar
  46. Young WC, Blumenstock JE, Fox EB, McCormick TH (2014) Detecting and classifying anomalous behavior in spatiotemporal network data. In: Proceedings of KDD workshop on learning about emergencies from social information (KDD-LESI 2014), pp 29–33Google Scholar
  47. Yuan J, Zheng Y, Xie X (2012) Discovering regions of different functions in a city using human mobility and pois. In: Proceedings of the ACM SIGKDD international conference on Knowledge discovery and data mining, pp 186–194Google Scholar
  48. Zhang T, Wang X, Li Z, Guo F, Ma Y, Chen W (2017) A survey of network anomaly visualization. Sc China Infor Sci 60(12):121101CrossRefGoogle Scholar
  49. Zhao Z, Cheng Z, Hong L, Chi EH (2015) Improving user topic interest profiles by behavior factorization. In: Proceedings of the international conference on world wide web. ACM, pp 1406–1416Google Scholar
  50. Zheng Y, Liu T, Wang Y, Zhu Y, Liu Y, Chang E (2014) Diagnosing New York city’s noises with ubiquitous data. In: Proceedings of the ACM international joint conference on pervasive and ubiquitous computing, pp 715–725Google Scholar

Copyright information

© The Author(s) 2018

Authors and Affiliations

  • Chaoguang Lin
    • 1
  • Qiuhan Zhu
    • 1
  • Shunan Guo
    • 1
  • Zhuochen Jin
    • 1
  • Yu-Ru Lin
    • 2
  • Nan Cao
    • 1
  1. 1.Intelligent Big Data Visualization LabTongji UniversityShanghaiChina
  2. 2.University of PittsburghPittsburghUSA

Personalised recommendations