An Introduction to Outlier Ensembles

  • Charu C. AggarwalEmail author
  • Saket Sathe


The outlier analysis problem has been widely studied by database, data mining, machine learning and statistical communities. Numerous algorithms have been proposed for this problem in recent years (Aggarwal, Outlier Detection in High Dimensional Data, [6]; Angiulli, Fast Outlier Detection in High Dimensional Spaces, [9]; Bay, Mining distance-based outliers in near linear time with randomization and a simple pruning rule, [11]; Breunig, LOF: Identifying Density-based Local Outliers, [14]; Knorr, Algorithms for Mining Distance-based Outliers in Large Datasets, [35]; Knorr, Finding Intensional Knowledge of Distance-Based Outliers, [36]; Jin, Mining top-n local outliers in large databases, [39]; Johnson, Fast computation of 2-dimensional depth contours, [40]; Papadimitriou, LOCI: Fast outlier detection using the local correlation integral, [53]; Ramaswamy, Efficient Algorithms for Mining Outliers from Large Data Sets, [55]).


Random Forest Outlier Detection Ensemble Method Subspace Cluster Combination Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    C. C. Aggarwal. A Human-Computer Interactive Method for Projected Clustering. IEEE Transactions on Knowledge and Data Engineering, 16(4), pp. 448–460, 2004.Google Scholar
  2. 2.
    C. C. Aggarwal. Outlier Ensembles: Position Paper, ACM SIGKDD Explorations, 14(2), pp. 49–58, December, 2012.Google Scholar
  3. 3.
    C. C. Aggarwal. Outlier Analysis, Second Edition, Springer, 2017.Google Scholar
  4. 4.
    C. C. Aggarwal, C. Reddy. Data Clustering: Algorithms and Applications, CRC Press, 2013.Google Scholar
  5. 5.
    C. C. Aggarwal and S. Sathe. Theoretical Foundations and Algorithms for Outlier Ensembles, ACM SIGKDD Explorations, 17(1), June 2015.Google Scholar
  6. 6.
    C. C. Aggarwal and P. S. Yu. Outlier Detection in High Dimensional Data, ACM SIGMOD Conference, 2001.Google Scholar
  7. 7.
    C. C. Aggarwal and P. S. Yu. Outlier Detection in Graph Streams. IEEE ICDE Conference, 2011.Google Scholar
  8. 8.
    C. C. Aggarwal, C. Procopiuc, J. Wolf, P. Yu, and J. Park. Fast Algorithms for Projected Clustering, ACM SIGMOD Conference, 1999.Google Scholar
  9. 9.
    F. Angiulli, C. Pizzuti. Fast Outlier Detection in High Dimensional Spaces, PKDD Conference, 2002.Google Scholar
  10. 10.
    D. Barbara, Y. Li, J. Couto, J.-L. Lin, and S. Jajodia. Bootstrapping a Data Mining Intrusion Detection System. Symposium on Applied Computing, 2003.Google Scholar
  11. 11.
    S. D. Bay and M. Schwabacher, Mining distance-based outliers in near linear time with randomization and a simple pruning rule, KDD Conf., 2003.Google Scholar
  12. 12.
    R. Bell and Y. Koren. Lessons from the Netflix prize challenge. ACM SIGKDD Explorations Newsletter, 9(2), pp. 75–79, 2007.Google Scholar
  13. 13.
    S. Bickel, T. Scheffer. Multi-view clustering. ICDM Conference, 2004.Google Scholar
  14. 14.
    M. Breunig, H.-P. Kriegel, R. Ng, and J. Sander. LOF: Identifying Density-based Local Outliers, ACM SIGMOD Conference, 2000.Google Scholar
  15. 15.
    L. Brieman. Bagging Predictors. Machine Learning, 24(2), pp. 123–140, 1996.Google Scholar
  16. 16.
    L. Brieman. Random Forests. Journal Machine Learning archive, 45(1), pp. 5–32, 2001.Google Scholar
  17. 17.
    R. Bryll, R. Gutierrez-Osuna, and F. Quek. Attribute Bagging: Improving Accuracy of Classifier Ensembles by using Random Feature Subsets. Pattern Recognition, 36(6), pp. 1291–1302, 2003.Google Scholar
  18. 18.
    P. Buhlmann. Bagging, subagging and bragging for improving some prediction algorithms, Recent advances and trends in nonparametric statistics, Elsevier, 2003.Google Scholar
  19. 19.
    V. Chandola, A. Banerjee, V. Kumar. Anomaly Detection: A Survey, ACM Computing Surveys, 2009.Google Scholar
  20. 20.
    N. Chawla, A. Lazarevic, L. Hall, and K. Bowyer. SMOTEBoost: Improving prediction of the minority class in boosting, PKDD, pp. 107–119, 2003.Google Scholar
  21. 21.
    J. Chen, S. Sathe, C. Aggarwal, and D. Turaga. Outlier Detection with Autoencoder Ensembles. SIAM Conference on Data Mining, 2017.Google Scholar
  22. 22.
    B. Clarke. Bayes Model Averaging and Stacking when Model Approximation Error cannot be Ignored, Journal of Machine Learning Research, pp 683–712, 2003.Google Scholar
  23. 23.
    P. Domingos. Bayesian Averaging of Classifiers and the Overfitting Problem. ICML Conference, 2000.Google Scholar
  24. 24.
    A. Emmott, S. Das, T. Dietteerich, A. Fern, and W. Wong. Systematic Construction of Anomaly Detection Benchmarks from Real Data. arXiv:1503.01158, 2015.
  25. 25.
    M. Fernandez-Delgado, E. Cernadas, S. Barro, and D. Amorim. Do we Need Hundreds of Classifiers to Solve Real World Classification Problems? The Journal of Machine Learning Research, 15(1), pp. 3133–3181, 2014.Google Scholar
  26. 26.
    Y. Freund and R. Schapire. A Decision-theoretic Generalization of Online Learning and Application to Boosting, Computational Learning Theory, 1995.Google Scholar
  27. 27.
    Y. Freund and R. Schapire. Experiments with a New Boosting Algorithm. ICML Conference, pp. 148–156, 1996.Google Scholar
  28. 28.
    J. Gao and P.-N. Tan. Converting output scores from outlier detection algorithms into probability estimates. ICDM Conference, 2006.Google Scholar
  29. 29.
    S. Guha, N. Mishra, G. Roy, and O. Schrijver. Robust Random Cut Forest Based Anomaly Detection On Streams. ICML Conference, pp. 2712–2721, 2016.Google Scholar
  30. 30.
    M. Grill and T. Pevny. Learning Combination of Anomaly Detectors for Security Domain. Computer Networks, 2016.Google Scholar
  31. 31.
    Z. He, S. Deng and X. Xu. A Unified Subspace Outlier Ensemble Framework for Outlier Detection, Advances in Web Age Information Management, 2005.Google Scholar
  32. 32.
    A. Hinneburg, D. Keim, and M. Wawryniuk. Hd-eye: Visual mining of high-dimensional data. IEEE Computer Graphics and Applications, 19:22–31, 1999.Google Scholar
  33. 33.
    T. K. Ho. Random decision forests. Third International Conference on Document Analysis and Recognition, 1995. Extended version appears as “The random subspace method for constructing decision forests” in IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(8), pp. 832–844, 1998.Google Scholar
  34. 34.
    T. K. Ho. Nearest Neighbors in Random Subspaces. Lecture Notes in Computer Science, Vol. 1451, pp. 640–648, Proceedings of the Joint IAPR Workshops SSPR’98 and SPR’98, 1998.
  35. 35.
    W. Jin, A. Tung, and J. Han. Mining top-n local outliers in large databases, ACM KDD Conference, 2001.Google Scholar
  36. 36.
    T. Johnson, I. Kwok, and R. Ng. Fast computation of 2-dimensional depth contours. ACM KDD Conference, 1998.Google Scholar
  37. 37.
    M. Joshi, V. Kumar, and R. Agarwal. Evaluating Boosting Algorithms to Classify Rare Classes: Comparison and Improvements. ICDM Conference, pp. 257–264, 2001.Google Scholar
  38. 38.
    F. Keller, E. Muller, K. Bohm. HiCS: High-Contrast Subspaces for Density-based Outlier Ranking, IEEE ICDE Conference, 2012.Google Scholar
  39. 39.
    E. Knorr, and R. Ng. Algorithms for Mining Distance-based Outliers in Large Datasets. VLDB Conference, 1998.Google Scholar
  40. 40.
    E. Knorr, and R. Ng. Finding Intensional Knowledge of Distance-Based Outliers. VLDB Conference, 1999.Google Scholar
  41. 41.
    A. Lazarevic, and V. Kumar. Feature Bagging for Outlier Detection, ACM KDD Conference, 2005.Google Scholar
  42. 42.
    F. T. Liu, K. N. Ting, and Z.-H. Zhou. On Detecting Clustered Anomalies using SCiForest. Machine Learning and Knowledge Discovery in Databases, pp. 274–290, Springer, 2010.Google Scholar
  43. 43.
    F. T. Liu, K. M. Ting, and Z.-H. Zhou. Isolation Forest. ICDM Conference, 2008. Extended version appears in: ACM Transactions on Knowledge Discovery from Data (TKDD), 6(1), 3, 2012.Google Scholar
  44. 44.
    L. M. Manevitz and M. Yousef. One-class SVMs for Document Classification, Journal of Machine Learning Research, 2: pp, 139–154, 2001.Google Scholar
  45. 45.
    B. Micenkova, B. McWilliams, and I. Assent. Learning Outlier Ensembles: The Best of Both Worlds Supervised and Unsupervised. ACM SIGKDD Workshop on Outlier Detection and Description, ODD, 2014.Google Scholar
  46. 46.
    B. Micenkova, B. McWilliams, and I. Assent. Learning Representations for Outlier Detection on a Budget. arXiv preprint arXiv:1507.08104, 2014.
  47. 47.
    F. Moosmann, B. Triggs, and F. Jurie. Fast Discriminative Visual Codebooks using Randomized Clustering Forests. Neural Information Processing Systems, pp. 985–992, 2006.Google Scholar
  48. 48.
    E. Muller, M. Schiffer, and T. Seidl. Statistical Selection of Relevant Subspace Projections for Outlier Ranking. ICDE Conference, pp, 434–445, 2011.Google Scholar
  49. 49.
    E. Muller, S. Gunnemann, I. Farber, and T. Seidl, Discovering multiple clustering solutions: Grouping objects in different views of the data, ICDM Conference, 2010.Google Scholar
  50. 50.
    E. Muller, S. Gunnemann, T. Seidl, and I. Farber. Tutorial: Discovering Multiple Clustering Solutions Grouping Objects in Different Views of the Data. ICDE Conference, 2012.Google Scholar
  51. 51.
    E. Muller, I. Assent, P. Iglesias, Y. Mulle, and K. Bohm. Outlier Ranking via Subspace Analysis in Multiple Views of the Data, ICDM Conference, 2012.Google Scholar
  52. 52.
    H. Nguyen, H. Ang, and V. Gopalakrishnan. Mining ensembles of heterogeneous detectors on random subspaces, DASFAA, 2010.Google Scholar
  53. 53.
    S. Papadimitriou, H. Kitagawa, P. Gibbons, and C. Faloutsos, LOCI: Fast outlier detection using the local correlation integral, ICDE Conference, 2003.Google Scholar
  54. 54.
    T. Pevny. Loda: Lightweight On-line Detector of Anomalies. Machine Learning, 102(2), pp. 275–304, 2016.Google Scholar
  55. 55.
    S. Ramaswamy, R. Rastogi, and K. Shim. Efficient Algorithms for Mining Outliers from Large Data Sets. ACM SIGMOD Conference, pp. 427–438, 2000.Google Scholar
  56. 56.
    S. Rayana and L. Akoglu. Less is More: Building Selective Anomaly Ensembles. ACM Transactions on Knowledge Disovery and Data Mining, 10(4), 42, 2016.Google Scholar
  57. 57.
    S. Rayana, W. Zhong, and L. Akoglu. Sequential Ensemble Learning for Outlier Detection: A Bias-Variance Perspective. IEEE ICDM Conference, 2016.Google Scholar
  58. 58.
    L. Rokach. Pattern classification using ensemble methods, World Scientific Publishing Company, 2010.Google Scholar
  59. 59.
    M. Salehi, X. Zhang, J. Bezdek, and C. Leckie. Smart Sampling: A Novel Unsupervised Boosting Approach for Outlier Detection. Australasian Joint Conference on Artificial Intelligence, Springer, pp. 469–481, 2016.
  60. 60.
    S. Sathe and C. Aggarwal. Subspace Outlier Detection in Linear Time with Randomized Hashing. ICDM Conference, 2016.Google Scholar
  61. 61.
    B. Scholkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson. Estimating the support of a high-dimensional distribution. Neural Computation, 13(7), pp. 1443–1472, 2001.Google Scholar
  62. 62.
    B. Scholkopf, R. C. Williamson, A. J. Smola, J. Shawe-Taylor, and J. C. Platt. Support-vector Method for Novelty Detection, Advances in Neural Information Processing Systems, 2000.Google Scholar
  63. 63.
    G. Seni, J. Elder, and R. Grossman. Ensemble Methods in Data Mining: Improving Accuracy through combining predictions. Morgan and Claypool, 2010.Google Scholar
  64. 64.
    P. Smyth and D. Wolpert. Linearly Combining Density Estimators via Stacking. Machine Learning Journal, 36, pp. 59–83, 1999.Google Scholar
  65. 65.
    A. Strehl and J. Ghosh. Cluster ensembles: A Knowledge Reuse Framework for Combining Multiple Partitions. Journal of Machine Learning Research, 3, pp. 583–617, 2001.Google Scholar
  66. 66.
    D. Tax and R. Duin. Combining One-Class Classifiers. Multiple Classifier Systems, pp. 299–308, 2001.Google Scholar
  67. 67.
    D. Tax and R. Duin. Support Vector Data Description. Machine learning, 54(1), 45-66, 2004.Google Scholar
  68. 68.
    D. Wolpert. Stacked Generalization, Neural Networks, 5(2), pp. 241–259, 1992.Google Scholar
  69. 69.
    H. Xiao, J. Gao, D. Turaga, L. Vu, and A. Biem. Temporal Multi-view Inconsistency Detection for Network Traffic Analysis. WWW Conference, pp. 455–465, 2015.Google Scholar
  70. 70.
    B. Zenko. Is Combining Classifiers Better than Selecting the Best One. Machine Learning, pp. 255–273, 2004.Google Scholar
  71. 71.
    Z.-H. Zhou. Ensemble Methods: Foundations and Algorithms. Chapman and Hall/CRC Press, 2012.Google Scholar
  72. 72.
    A. Zimek, M. Gaudet, R. Campello, J. Sander. Subsampling for efficient and effective unsupervised outlier detection ensembles, KDD Conference, 2013.Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.IBM T. J. Watson Research CenterYorktown HeightsUSA

Personalised recommendations