Abstract
Outlier Detection is a key technique to discover abnormal patterns from mass data. Pivot based metric space outlier detection algorithm is designed to improve the cutoff value of outlier degree faster and terminate the detection program earlier, for saving detection time. However, due to the lack of efficient pivot selection method, the performance of existing related algorithms is not as good as it might have been. In this paper, we propose an adaptive cutoff distance based density peak pivot selection algorithm (ADP) to get a suitable pivot quickly. Moreover, we also develop an improved outlier detection algorithm based on ADP algorithm (ADPOD). Experimental results indicate that ADPOD can effectively overcome the existing relate algorithm, and achieves a 2.67 speed up over it on average and, in certain cases, up to 8.06. The distance calculation times are reduced by 53.39% on average and up to 94.84%, as well as acceptable indexing time and the universality of metric space.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kriegel, B.H., Kröger, P., Zimek, A.: Outlier detection techniques. In: Tutorial at PAKDD (2010)
Aggarwal, C.C.: An introduction to data mining. In: Aggarwal, C.C. (eds.) Data Mining, pp. 1–26. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-14142-8_1
Zimek, A., Schubert, E.: Outlier detection. In: Liu, L., Özsu, M.T. (eds.) Encyclopedia of Database Systems, pp. 1–5. Springer, New York (2017). https://doi.org/10.1007/978-1-4899-7993-3_80719-1
Hawkins, D.M.: Identification of Outliers, vol. 11. Springer, Dordrecht (1980). https://doi.org/10.1007/978-94-015-3994-4
Othman, Z.A., et al.: Rough outlier method for network intrusion detection. Int. J. Inf. Process. Manag. 4(7), 39–50 (2013)
Srimani, P.D.P.K., Koti, M.S.: Outlier mining in medical databases by using statistical methods. Int. J. Eng. Sci. Technol. 4(1), 239–246 (2012)
Djenouri, Y., Zimek, A.: Outlier detection in urban traffic data. In: 2018 Proceedings of the 8th International Conference on Web Intelligence, Mining and Semantics, pp. 1–12. ACM, Novi Sad (2018)
Knorr, E.M., Ng, R.T., Tucakov, V.: Distance-based outliers: algorithms and applications. VLDB Int. J. Very Large Data Bases 8(3–4), 237–253 (2000)
Chaudhary, A., Szalay, A.S., Moore, A.W.: Very fast outlier detection in large multidimensional data sets. In: Proceedings of Data Mining and Knowledge Discovery (2002)
Ren, D., et al.: A vertical distance-based outlier detection method with local pruning. In: 2004 Proceedings of the 13th ACM International Conference on Information and Knowledge Management. ACM (2004)
Pillutla, M.R., et al.: LSH based outlier detection and its application in distributed setting. In: 2011 Proceedings of the 20th ACM International Conference on Information and Knowledge Management. ACM (2011)
Wang, Y., Parthasarathy, S., Tatikonda, S.: Locality sensitive outlier detection: a ranking driven approach. In: 2011 IEEE 27th International Conference on Data Engineering. IEEE (2011)
Wang, X.-T., Shen, D.-R., Bai, M.: BOD: an efficient algorithm for distributed outlier detection. Chin. J. Comput. 39(1), 36–51 (2016)
Yan, Y., Cao, L., Rundensteiner, E.: Scalable top-n local outlier detection. In: 2017 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1235–1244 (2017)
Bay, S.D., Schwabacher, M.: Mining distance-based outliers in near linear time with randomization and a simple pruning rule. In: 2003 Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, DC, USA (2003)
Bhaduri, K., Matthews, B.L., Giannella, C.R.: Algorithms for speeding up distance-based outlier detection. In: 2011 Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, USA (2011)
UCI Machine Learning Repository: Data Sets. http://archive.ics.uci.edu/ml/datasets.html. Accessed 1 Sept 2018
Acknowledgment
This research is supported by NSF-China (No. 61802062 and 61802063), and Scientific Research Foundation of Foshan University (No. gg07027).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Xu, H., Sun, F., Tan, L., Huang, W. (2019). Adaptive Cutoff Distance Based Density Peak Pivot for Metric Space Outlier Detection. In: Sun, F., Liu, H., Hu, D. (eds) Cognitive Systems and Signal Processing. ICCSIP 2018. Communications in Computer and Information Science, vol 1006. Springer, Singapore. https://doi.org/10.1007/978-981-13-7986-4_35
Download citation
DOI: https://doi.org/10.1007/978-981-13-7986-4_35
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-7985-7
Online ISBN: 978-981-13-7986-4
eBook Packages: Computer ScienceComputer Science (R0)