Advertisement

Anomaly Detection and Diagnosis for Container-Based Microservices with Performance Monitoring

  • Qingfeng Du
  • Tiandi Xie
  • Yu He
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11337)

Abstract

With emerging container technologies, such as Docker, microservices-based applications can be developed and deployed in cloud environment much agiler. The dependability of these microservices becomes a major concern of application providers. Anomalous behaviors which may lead to unexpected failures can be detected with anomaly detection techniques. In this paper, an anomaly detection system (ADS) is designed to detect and diagnose the anomalies in microservices by monitoring and analyzing real-time performance data of them. The proposed ADS consists of a monitoring module that collects the performance data of containers, a data processing module based on machine learning models and a fault injection module integrated for training these models. The fault injection module is also used to assess the anomaly detection and diagnosis performance of our ADS. Clearwater, an open source virtual IP Multimedia Subsystem, is used for the validation of our ADS and experimental results show that the proposed ADS works well.

Keywords

Anomaly detection Microservices Performance monitoring Machine learning 

References

  1. 1.
    Singh, V., et al.: Container-based microservice architecture for cloud applications. In: Computing, Communication and Automation (ICCCA) (2017)Google Scholar
  2. 2.
    Sauvanaud, C., et al.: Anomaly detection and diagnosis for cloud services: practical experiments and lessons learned. J. Syst. Softw. 139, 84–106 (2018)CrossRefGoogle Scholar
  3. 3.
    Rusek, M., Dwornicki, G., Orłowski, A.: A decentralized system for load balancing of containerized microservices in the cloud. In: Świątek, J., Tomczak, J.M. (eds.) ICSS 2016. AISC, vol. 539, pp. 142–152. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-48944-5_14CrossRefGoogle Scholar
  4. 4.
    Kratzke, N.: About microservices, containers and their underestimated impact on network performance. arXiv preprint arXiv:1710.04049(2017) (2017)
  5. 5.
    Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Computing Surveys (2009)Google Scholar
  6. 6.
    Wang, T., Zhang, W., Ye, C., et al.: FD4C: automatic fault diagnosis framework for web applications in cloud computing. IEEE Trans. Syst. Man Cybern.: Syst. 46(1), 61–75 (2016)CrossRefGoogle Scholar
  7. 7.
    Amaral, M., Polo, J., et al.: Performance evaluation of microservices architectures using containers. In: 2015 IEEE 14th International Symposium on Network Computing and Applications (NCA), pp. 27–34. IEEE (2015)Google Scholar
  8. 8.
    Ferreira, A., Felter, W., et al.: An updated performance comparison of virtual machines and Linux containers. Technical Report RC25482 (AUS1407-001). IBM (2014)Google Scholar
  9. 9.
    Kjallman, J., Morabito, R., Komu, M.: Hypervisors vs. lightweight virtualization: a performance comparison. In: IEEE International Conference on Cloud Engineering (2015)Google Scholar
  10. 10.
    Zheng, Z., Zhang, Y., Lyu, M.R.: An online performance prediction framework for service-oriented systems. IEEE Trans. Syst. Man Cybern. 44, 1169–1181 (2014)CrossRefGoogle Scholar
  11. 11.
    Mi, H., Wang, H., et al.: Toward fine-grained, unsupervised, scalable performance diagnosis for production cloud computing systems. IEEE Trans. Parallel Distrib. Syst. 24(6), 1245–1255 (2013)CrossRefGoogle Scholar
  12. 12.
    Zhang, S., Pattipati, K.R., et al.: Dynamic coupled fault diagnosis with propagation and observation delays. IEEE Trans. Syst. Man Cybern.: Syst. 43(6), 1424–1439 (2013)CrossRefGoogle Scholar
  13. 13.
    Pahl, C.: Containerization and the PaaS cloud. IEEE Cloud Comput. 2, 24–31 (2015)CrossRefGoogle Scholar
  14. 14.
    Liao, W.T.: Clustering of time series data–a survey. Pattern Recogn. 38(11), 1857–1874 (2005)CrossRefGoogle Scholar
  15. 15.
    Chen, Y., Keogh, E., et al.: The UCR time series classification archive, July 2015. www.cs.ucr.edu/~eamonn/time_series_data/
  16. 16.
    Clearwater: Project clearwater. http://www.projectclearwater.org/

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Software EngineeringTongji UniversityShanghaiChina

Personalised recommendations