Skip to main content
SpringerLink
Log in
Book cover

Anomaly-Detection and Health-Analysis Techniques for Core Router Systems pp 1–22Cite as

Introduction

  • Chapter
  • First Online:

  • 300 Accesses

Abstract

In this chapter, concepts of IP network and its hierarchy are first introduced. Detailed background of core routers in modern telecommunication systems is then described, including both hardware and software advancement. A wide range of faults in routers are also discussed to show why they are becoming more difficult to detect, diagnose and repair in time. Next, we discuss various prior work on anomaly detection, including their advantages and disadvantages. Finally, we summarize current technique challenges and the overall motivation of this book.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. P. Veitch et al., Integrating core BGP/MPLS networks. Internet Protoc. J. 13(4), 18–31 (2010)

    MathSciNet  Google Scholar 

  2. C.N. Academy, Connecting Networks Companion Guide (Pearson Education, London, 2014)

    Google Scholar 

  3. V. Antonenko, R. Smelyanskiy, Global network modelling based on mininet approach, in Proceedings of the Second ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (2013), pp. 145–146

    Google Scholar 

  4. L. Wayne, LAN Switching and Wireless, CCNA Exploration Companion Guide (Pearson Education India, Delhi, 2008)

    Google Scholar 

  5. R. Giladi, Network Processors: Architecture, Programming, and Implementation (Morgan Kaufmann, San Francisco, 2008)

    Google Scholar 

  6. M. Médard, S.S. Lumetta, Network reliability and fault tolerance, in Encyclopedia of Telecommunications (Wiley, New York, 2003)

    Google Scholar 

  7. S. Jin, F. Ye, Z. Zhang, K. Chakrabarty, X. Gu, Efficient board-level functional fault diagnosis with missing syndromes. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 35(6), 985–998 (2016)

    Article  Google Scholar 

  8. R. Isermann, Fault-Diagnosis Systems: An Introduction from Fault Detection to Fault Tolerance (Springer Science & Business Media, Berlin, 2006)

    Book  Google Scholar 

  9. B. Schroeder, G.A. Gibson, A large-scale study of failures in high-performance computing systems. IEEE Trans. Dependable Secure Comput. 7, 337–350 (2010)

    Article  Google Scholar 

  10. P.K. Patra, H. Singh, G. Singh, Fault tolerance techniques and comparative implementation in cloud computing. Int. J. Comput. Appl. 64, 37–41 (2013)

    Google Scholar 

  11. P.A. Lee, T. Anderson, Fault Tolerance: Principles and Practice, vol. 3 (Springer Science & Business Media, Berlin, 2012)

    MATH  Google Scholar 

  12. F. Ye, Z. Zhang, K. Chakrabarty, X. Gu, Information-theoretic syndrome and root-cause analysis for guiding board-level fault diagnosis, in Proceedings of IEEE European Test Symposium (ETS) (2013), pp. 1–6

    Google Scholar 

  13. C. Wang, F. Mueller, C. Engelmann, S.L. Scott, Proactive process-level live migration in HPC environments, in Proceedings of the 2008 ACM/IEEE Conference on Supercomputing (2008), p. 43

    Google Scholar 

  14. F. Ye, Z. Zhang, K. Chakrabarty, X. Gu, Board-level functional fault diagnosis using artificial neural networks, support-vector machines, and weighted-majority voting. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 32, 723–736 (2013)

    Article  Google Scholar 

  15. F. Ye, Z. Zhang, K. Chakrabarty, X. Gu, Knowledge discovery and knowledge transfer in board-level functional fault diagnosis, in Proceedings of IEEE International Test Conference (ITC) (2014), pp. 1–10

    Google Scholar 

  16. C.-K. Hsu et al., Test data analytics–exploring spatial and test-item correlations in production test data, in Proceedings of IEEE International Test Conference (ITC) (2013), pp. 1–10

    Google Scholar 

  17. S. Tanwir et al., Information-theoretic and statistical methods of failure log selection for improved diagnosis, in Proceedings of ITC, 2015

    Google Scholar 

  18. A. Gainaru, F. Cappello, M. Snir, W. Kramer, Fault prediction under the microscope: a closer look into hpc systems, in Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (2012), p. 77

    Google Scholar 

  19. V. Kumar, Parallel and distributed computing for cybersecurity. IEEE Distrib. Syst. Online 6, 1–9 (2005)

    Article  Google Scholar 

  20. C. Spence, L. Parra, P. Sajda, Detection, synthesis and compression in mammographic image analysis with a hierarchical image probability model, in Proceedings of the IEEE Workshop on Mathematical Methods in Biomedical Image, 2001

    Google Scholar 

  21. E. Aleskerov, B. Freisleben, B. Rao, Cardwatch: a neural network based database mining system for credit card fraud detection, in Proceedings of the IEEE/IAFE Computational Intelligence for Financial Engineering (1997), pp. 220–226

    Google Scholar 

  22. A. Patcha, J.-M. Park, An overview of anomaly detection techniques: existing solutions and latest technological trends. Comput. Netw. 51, 3448–3470 (2007)

    Article  Google Scholar 

  23. V. Chandola, A. Banerjee, V. Kumar, Anomaly detection: a survey. ACM Comput. Surv. 15:1–15:58, 15 (2009)

    Google Scholar 

  24. P. Gogoi et al., A survey of outlier detection methods in network anomaly identification. Comput. J. 54, 570–588 (2011)

    Article  Google Scholar 

  25. S. Agrawal, J. Agrawal, Survey on anomaly detection using data mining Techniques. Proc. Comput. Sci. 60, 708–713 (2015)

    Article  Google Scholar 

  26. B. Al-Musawi et al., BGP anomaly detection techniques: a survey. IEEE Commun. Surv. 19, 377–396 (2017)

    Article  Google Scholar 

  27. S. Konishi, G. Kitagawa, Information Criteria and Statistical Modeling (Springer Science & Business Media, Berlin, 2008)

    Book  MATH  Google Scholar 

  28. M.H. Bhuyan et al., Network anomaly detection: methods, systems and tools. IEEE Commun. Surv. Tutorials 16, 303–336 (2014)

    Article  Google Scholar 

  29. E. Eskin, Anomaly detection over noisy data using learned probability distributions, in Proceedings of the Seventeenth International Conference on Machine Learning (2000), pp. 255–262

    Google Scholar 

  30. J. Aldrich, R.a. fisher and the making of maximum likelihood 1912–1922. Stat. Sci. 12, 162–176 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  31. E. Grafarend, Linear and Nonlinear Models: Fixed Effects, Random Effects, and Mixed Models (Walter de Gruyter, Berlin, 2006)

    Google Scholar 

  32. H.E. Solberg, A. Lahti, Detection of outliers in reference distributions: performance of Horn’s algorithm. Clin. Chem. 51, 2326–2332 (2005)

    Article  Google Scholar 

  33. M. Hubert, E. Vandervieren, An adjusted boxplot for skewed distributions. Comput. Stat. Data Anal. 52, 5186–5201 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  34. K. Yamanishi et al., On-line unsupervised outlier detection using finite mixtures with discounting learning algorithms. Data Min. Knowl. Discov. 8, 275–300 (2004)

    Article  MathSciNet  Google Scholar 

  35. P.H. Torr, D.W. Murray, Outlier detection and motion segmentation, in Sensor Fusion VI, vol. 2059 (1993), pp. 432–444

    Google Scholar 

  36. P. Galeano, D. Peña, R.S. Tsay, Outlier detection in multivariate time series by projection pursuit. J. Am. Stat. Assoc. 101, 654–669 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  37. M. Thottan, C. Ji, Anomaly detection in IP networks. IEEE Trans. Signal Process. 51(8), 2191–2204 (2003)

    Article  Google Scholar 

  38. V. Vapnik, An overview of statistical learning theory. IEEE Trans. Neural Netw. 10(5), 988–999 (1999)

    Article  Google Scholar 

  39. M. Celenk, T. Conley, J. Willis, J. Graham, Predictive network anomaly detection and visualization. IEEE Trans. Inf. Forensics Secur. 5, 288–299 (2010)

    Article  Google Scholar 

  40. P.J. Rousseeuw, A.M. Leroy, Robust Regression and Outlier Detection, vol. 589 (Wiley, New York, 2005)

    MATH  Google Scholar 

  41. E. Eskin, A. Arnold, M. Prerau, L. Portnoy, S. Stolfo, A geometric framework for unsupervised anomaly detection, in Applications of Data Mining in Computer Security (Springer, Berlin, 2002), pp. 77–101

    Book  Google Scholar 

  42. T. Fawcett, F. Provost, Activity monitoring: noticing interesting changes in behavior, in Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (1999), pp. 53–62

    Google Scholar 

  43. L.L. Ho, C.J. Macey, R. Hiller, A distributed and reliable platform for adaptive anomaly detection in IP networks, in International Workshop on Distributed Systems: Operations and Management (1999), pp. 33–46

    Chapter  Google Scholar 

  44. B.W. Silverman, Density Estimation for Statistics and Data Analysis (Routledge, London, 2018)

    Book  Google Scholar 

  45. E. Parzen, On estimation of a probability density function and mode. Ann. Math. Stat. 33, 1065–1076 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  46. D.-Y. Yeung, C. Chow, Parzen-window network intrusion detectors, in 16th International Conference on Pattern Recognition, 2002. Proceedings, vol. 4 (2002), pp. 385–388

    Google Scholar 

  47. C.S. Teh, C.P. Lim, Monitoring the formation of kernel-based topographic maps in a hybrid SOM-kMER model. IEEE Trans. Neural Netw. 17, 1336–1341 (2006)

    Article  Google Scholar 

  48. G. Bloch et al., Reduced-size kernel models for nonlinear hybrid system Identification. IEEE Trans. Neural Netw. 22, 2398–2405 (2011)

    Article  Google Scholar 

  49. W. Lee, D. Xiang, Information-theoretic measures for anomaly detection, in 2001 IEEE Symposium on Security and Privacy, 2001. S&P 2001. Proceedings (2001), pp. 130–143

    Google Scholar 

  50. L. Ming, P. Vitányi, An Introduction to Kolmogorov Complexity and Its Applications (Springer, Heidelberg, 1997)

    MATH  Google Scholar 

  51. Z. He, S. Deng, X. Xu, J.Z. Huang, A fast greedy algorithm for outlier mining, in Advances in Knowledge Discovery and Data Mining (2006), pp. 567–576

    Chapter  Google Scholar 

  52. H. Motulsky, Intuitive Biostatistics: A Nonmathematical Guide to Statistical Thinking (Oxford University Press, Oxford, 2013)

    Google Scholar 

  53. P.-N. Tan, M. Steinbach, V. Kumar et al., Introduction to Data Mining (Pearson Education India, Delhi, 2006)

    Google Scholar 

  54. O. Maimon, L. Rokach, Data Mining and Knowledge Discovery Handbook (Springer, New York, 2005)

    Book  MATH  Google Scholar 

  55. G. Münz, S. Li, G. Carle, Traffic anomaly detection using k-means clustering, in GI/ITG Workshop MMBnet, 2007

    Google Scholar 

  56. I. Syarif, A. Prugel-Bennett, G. Wills, Data mining approaches for network intrusion detection: from dimensionality reduction to misuse and anomaly detection. J. Inf. Technol. Rev. 3, 70–83 (2012)

    Google Scholar 

  57. M. Ester, H.-P. Kriegel, J. Sander, X. Xu, et al., A density-based algorithm for discovering clusters in large spatial databases with noise, in Kdd, vol. 96, no. 34 (1996), pp. 226–231. https://scholar.googleusercontent.com/scholar.bib?q=info:-KybkyxcGYIJ:scholar.google.com/&output=citation&scisdr=CgXKeW0REOXD7i92Y3Q:AAGBfm0AAAAAXe1ze3R8hFXHZTU1cF2vgN_fTvfsj7n8&scisig=AAGBfm0AAAAAXe1ze25z3yqnbvDqkrKyrFQlrHSHZvKk&scisf=4&ct=citation&cd=-1&hl=en

  58. S. Guha, R. Rastogi, K. Shim, Rock: a robust clustering algorithm for categorical attributes, in 15th International Conference on Data Engineering, 1999. Proceedings (1999), pp. 512–521

    Google Scholar 

  59. L. Ertöz, M. Steinbach, V. Kumar, Finding topics in collections of documents: a shared nearest neighbor approach, in Clustering and Information Retrieval (Springer, 2004), pp. 83–103. https://scholar.googleusercontent.com/scholar.bib?q=info:6_FGDFfMgdsJ:scholar.google.com/&output=citation&scisdr=CgXKeW0REOXD7i92rAo:AAGBfm0AAAAAXe1ztApZ5-i7PHwRAvc7Jl9hQS2iKb77&scisig=AAGBfm0AAAAAXe1ztPaMM5l8A2fOfe5_nNBur3GiTTPt&scisf=4&ct=citation&cd=-1&hl=en

  60. D. Yu, G. Sheikholeslami, A. Zhang, Findout: finding outliers in very large datasets, in Knowledge and Information Systems, vol. 4 (Springer, London, 2002), pp. 387–412

    Article  Google Scholar 

  61. M.M. Breunig et al., Lof: identifying density-based local outliers, in ACM Sigmod Record, vol. 29 (ACM, New York, 2000), pp. 93–104

    Article  Google Scholar 

  62. L. Ertoz et al., Minds-Minnesota intrusion detection system, in Next Generation Data Mining (MIT Press, Boston, 2004), pp. 199–218

    Google Scholar 

  63. Z. He, X. Xu, S. Deng, Discovering cluster-based local outliers. Pattern Recogn. Lett. 24, 1641–1650 (2003)

    Article  MATH  Google Scholar 

  64. E. Alpaydin, Introduction to Machine Learning (MIT Press, Cambridge, 2014)

    MATH  Google Scholar 

  65. S.J. Russell, P. Norvig, Artificial Intelligence: A Modern Approach (Pearson Education Limited, Malaysia/Prentice Hall, Englewood Cliffs, 2016). https://scholar.googleusercontent.com/scholar.bib?q=info:I5nM5aK3CioJ:scholar.google.com/&output=citation&scisdr=CgXKeW0REOXD7i9xxp0:AAGBfm0AAAAAXe103p0rY6JwTpsa52HsHX_I0dv1R3fl&scisig=AAGBfm0AAAAAXe103pLBwolLgLFeP9K3udaS0GJ3tkhr&scisf=4&ct=citation&cd=-1&hl=en

  66. C. Kruegel, D. Mutz, W. Robertson, F. Valeur, Bayesian event classification for intrusion detection, in Computer Security Applications Conference, 2003. Proceedings. 19th Annual (2003), pp. 14–23

    Google Scholar 

  67. D. Janakiram, V. Adi Mallikarjuna Reddy, A. Kumar, Outlier detection in wireless sensor networks using Bayesian belief networks, in International Conference on Communication System Software and Middleware (2006), pp. 1–6

    Google Scholar 

  68. G. Rätsch, S. Mika, B. Scholkopf, K.-R. Müller, Constructing boosting algorithms from SVMs: an application to one-class classification. IEEE Trans. Pattern Anal. Mach. Intell. 24, 1184–1199 (2002)

    Article  Google Scholar 

  69. K.A. Heller et al., One class support vector machines for detecting anomalous windows registry accesses, in Proceedings of the Workshop on Data Mining for Computer Security, vol. 9, 2003

    Google Scholar 

  70. A. Lazarevic et al., A comparative study of anomaly detection schemes in network intrusion detection, in Proceedings of the 2003 SIAM International Conference on Data Mining (2003), pp. 25–36

    Google Scholar 

  71. S. Mukkamala, G. Janoski, A. Sung, Intrusion detection using neural networks and support vector machines, in Proceedings of the International Joint Conference on Neural Networks, vol. 2 (2002), pp. 1702–1707

    Google Scholar 

  72. C. Cortes, V. Vapnik, Support-vector networks. Mach. Learn. 20, 273–297 (1995)

    MATH  Google Scholar 

  73. D.M.J. Tax, One-class classification: concept-learning in the absence of counter-examples. PhD thesis, Delft University of Technology, 2001

    Google Scholar 

  74. W. Lee et al., A data mining and CIDF based approach for detecting novel and distributed intrusions, in International Workshop on Recent Advances in Intrusion Detection (2000), pp. 49–65

    Google Scholar 

  75. W. Lee, S.J. Stolfo, K.W. Mok, Adaptive intrusion detection: a data mining approach. Artif. Intell. Rev. 14, 533–567 (2000)

    Article  MATH  Google Scholar 

  76. J. Quinlan, Induction of decision trees. Mach. Learn. 1, 81–106 (1986)

    Google Scholar 

  77. J.R. Quinlan et al., Bagging, boosting, and c4. 5, in AAAI/IAAI, Vol. 1 (1996), pp. 725–730

    Google Scholar 

  78. R.L. Lawrence, A. Wright, Rule-based classification systems using classification and regression tree (cart) analysis. Photogramm. Eng. Remote Sens. 67, 1137–1142 (2001)

    Google Scholar 

  79. N.B. Amor, S. Benferhat, Z. Elouedi, Naive Bayes vs decision trees in intrusion detection systems, in Proceedings of the 2004 ACM Symposium on Applied Computing (2004), pp. 420–424

    Google Scholar 

  80. L.V. Fausett et al., Fundamentals of Neural Networks: Architectures, Algorithms, and Applications, vol. 3 (Prentice-Hall, Englewood Cliffs, 1994)

    MATH  Google Scholar 

  81. M. Amini, R. Jalili, H.R. Shahriari, Rt-unnid: a practical solution to real-time network-based intrusion detection using unsupervised neural networks. Comput. Secur. 25, 459–468 (2006)

    Article  Google Scholar 

  82. J. Schmidhuber, Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  Google Scholar 

  83. M.S. alDosari, Unsupervised anomaly detection in sequences using long short term memory recurrent neural networks, Master’s thesis, 2016

    Google Scholar 

  84. N. Shone et al., A deep learning approach to network intrusion detection. IEEE Trans. Emerg. Top. Comput. Intell. 2, 41–50 (2018)

    Article  Google Scholar 

  85. W. Wang et al., Hast-ids: learning hierarchical spatial-temporal features using deep neural networks to improve intrusion detection. IEEE Access 6, 1792–1806 (2018)

    Article  Google Scholar 

  86. A. Graves et al., Speech recognition with deep recurrent neural networks, in International Conference on Acoustics, Speech and Signal Processing (2013), pp. 6645–6649

    Google Scholar 

  87. G.E. Hinton, R.R. Salakhutdinov, Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Hardware, Nvidia Corporation, Sunnyvale, CA, USA

    Shi Jin

  2. Futurewei Technologies, Santa Clara, CA, USA

    Zhaobo Zhang & Xinli Gu

  3. Electrical and Computer Engineering, Duke University, Durham, NC, USA

    Krishnendu Chakrabarty

Authors
  1. Shi Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhaobo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Krishnendu Chakrabarty
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinli Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Jin, S., Zhang, Z., Chakrabarty, K., Gu, X. (2020). Introduction. In: Anomaly-Detection and Health-Analysis Techniques for Core Router Systems. Springer, Cham. https://doi.org/10.1007/978-3-030-33664-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-33664-6_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-33663-9

  • Online ISBN: 978-3-030-33664-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation