Abstract
Based on the widely known kernel density estimator of the probability function, a new algorithm is proposed in order to detect outliers and to provide a robust estimation of location and scatter. With the help of the Gaussian Transform, a robust weighted kernel estimation of the density probability function is calculated, referring to the whole of the data, including the outliers. In the next step, the data points having the smallest values according to the robust pdf are removed as the least probable to belong to the clean data. The program based on this algorithm is more accurate even on greatly correlated outliers with the data, and even with outliers with small Euclidean distance from the data. In case the data have many variables, we can use Principal Component algorithm by (Introduction to Multivariate Statistical Analysis in Chemometrics. CRC press, 2008) [1] with the same efficiency in the detection of outliers.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Varmuza, K., Filzmoser, P.: Introduction to Multivariate Statistical Analysis in Chemometrics. CRC press (2008)
Hawkins, D.M.: Identification of Outliers. Springer-Science and Business Media, B.V. (1980)
Maronna, R.A., Yohai, V.J.: The behavior of the Stahel-Don oho robust multivariate estimator. J. Am. Stat. Assoc. 90(429), 330–341 (1995)
Rousseeuw, P.J., Van Driessen, K.: A fast algorithm for the minimum covariance determinant estimator. Technometr. 41(3), 212–223 (1999)
Filzmoser, P., Hron, K., Reimann, C.: Principal component analysis for compositional data with outliers, Vienna University of Technology, Forschungsbericht, 2007
Kernel Density Estimation: Wikipedia.org, http://wikipedia.org/wiki/Kernel_density_estimation
Maronna, R.A., Zamar, R.H.: Robust estimates of location and dispersion for high-dimensional datasets. Technometr. 44(4) (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media New York
About this paper
Cite this paper
Pavlidou, M., Zioutas, G. (2014). Kernel Density Outlier Detector. In: Akritas, M., Lahiri, S., Politis, D. (eds) Topics in Nonparametric Statistics. Springer Proceedings in Mathematics & Statistics, vol 74. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-0569-0_22
Download citation
DOI: https://doi.org/10.1007/978-1-4939-0569-0_22
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4939-0568-3
Online ISBN: 978-1-4939-0569-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)