Advertisement

Detecting Anomaly in Cloud Platforms Using a Wavelet-Based Framework

  • David O’Shea
  • Vincent C. EmeakarohaEmail author
  • Neil Cafferkey
  • John P. Morrison
  • Theo Lynn
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 740)

Abstract

Cloud computing enables the delivery of compute resources as services in an on-demand fashion. The reliability of these services is of significant importance to their consumers. The presence of anomaly in Cloud platforms can put their reliability into question, since an anomaly indicates deviation from normal behaviour. Monitoring enables efficient Cloud service provisioning management; however, most of the management efforts are focused on the performance of the services and little attention is paid to detecting anomalous behaviour from the gathered monitoring data. In addition, the existing solutions for detecting anomaly in Clouds lacks a multi-dimensional approach. In this chapter, we present a wavelet-based anomaly detection framework that is capable of analysing multiple monitored metrics simultaneously to detect anomalous behaviour. It operates in both frequency and time domains in analysing monitoring data that represents system behaviour. The framework is first trained using over seven days worth of historical monitoring data to identify healthy behaviour. Based on this training, anomalous behaviour can be detected as deviations from the healthy system. The effectiveness of the proposed framework was evaluated based on a Cloud service deployment use-case scenario that produced both healthy and anomalous behaviour.

Keywords

Multi-dimensional anomaly detection Wavelet transformation Cloud monitoring Data analysis Cloud computing 

Notes

Acknowledgements

The research work described in this paper was supported by the Irish Centre for Cloud Computing and Commerce, an Irish national Technology Centre funded by Enterprise Ireland and the Irish Industrial Development Authority.

References

  1. 1.
    Agarwal, S., Mozafari, B., Panda, A., Milner, H., Madden, S., Stoica, I.: BlinkDB: queries with bounded errors and bounded response times on very large data. In: Proceedings of the 8th ACM European Conference on Computer Systems, pp. 29–42. ACM (2013)Google Scholar
  2. 2.
    Agarwala, S., Alegre, F., Schwan, K., Mehalingham, J.: E2EProf: automated end-to-end performance management for enterprise systems. In: 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2007, pp. 749–758, June 2007Google Scholar
  3. 3.
    Albanese, D., Visintainer, R., Merler, S., Riccadonna, S., Jurman, G., Furlanello, C.: mlpy: machine learning Python (2012). http://mlpy.sourceforge.net/. Accessed 22 Feb 2016
  4. 4.
    Apache Software Foundation. Apache JMeter (2016). http://jmeter.apache.org/. Accessed 06 Jan 2016
  5. 5.
    Bahl, P., Chandra, R., Greenberg, A., Kandula, S., Maltz, D., Zhang, M.: Towards highly reliable enterprise network services via inference of multi-level dependencies. In: SIGCOMM. Association for Computing Machinery Inc., August 2007Google Scholar
  6. 6.
    Bakhtazad, A., Palazoglu, A., Romagnoli, J.A.: Detection and classification of abnormal process situations using multidimensional wavelet domain hidden Markov trees. Comput. Chem. Eng. 24(2), 769–775 (2000)CrossRefGoogle Scholar
  7. 7.
    Buzen, J.P., Shum, A.W.: MASF - multivariate adaptive statistical filtering. In: International CMG Conference, pp. 1–10 (1995)Google Scholar
  8. 8.
    Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 15:1–15:58 (2009)CrossRefGoogle Scholar
  9. 9.
    Doelitzscher, F., Knahl, M., Reich, C., Clarke, N.: Anomaly detection in IaaS clouds. In: 2013 IEEE 5th International Conference on Cloud Computing Technology and Science (CloudCom), pp. 387–394, December 2013Google Scholar
  10. 10.
    Emeakaroha, V.C., Brandic, I., Maurer, M., Dustdar, S.: Low level metrics to high level SLAs - LoM2HiS framework: bridging the gap between monitored metrics and SLA parameters in cloud environments. In: 2010 International Conference on High Performance Computing and Simulation (HPCS), pp. 48–54, July 2010Google Scholar
  11. 11.
    Emeakaroha, V.C., Netto, M.A.S., Calheiros, R.N., Brandic, I., Buyya, R., De Rose, C.A.F.: Towards autonomic detection of SLA violations in cloud infrastructures. Future Gener. Comput. Syst. 28(7), 1017–1029 (2012)CrossRefGoogle Scholar
  12. 12.
    Fatema, K., Emeakaroha, V.C., Healy, P.D., Morrison, J.P., Lynn, T.: A survey of cloud monitoring tools: taxanomy, capabilities and objectives. J. Parallel Distrib. Comput. 74, 2918–2933 (2014)CrossRefGoogle Scholar
  13. 13.
    Frigo, M.: A fast Fourier transform compiler. ACM Sigplan Not. 34, 169–180 (1999). ACMCrossRefGoogle Scholar
  14. 14.
    Gander, M., Felderer, M., Katt, B., Tolbaru, A., Breu, R., Moschitti, A.: Anomaly detection in the cloud: detecting security incidents via machine learning. In: Moschitti, A., Plank, B. (eds.) Trustworthy Eternal Systems via Evolving Software, Data and Knowledge, pp. 103–116. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  15. 15.
    Guan, Q., Fu, S.: Adaptive anomaly identification by exploring metric subspace in cloud computing infrastructures. In: 2013 IEEE 32nd International Symposium on Reliable Distributed Systems (SRDS), pp. 205–214, September 2013Google Scholar
  16. 16.
    Guan, Q., Fu, S.: Wavelet-based multi-scale anomaly identification in cloud computing systems. In: 2013 IEEE Global Communications Conference (GLOBECOM), pp. 1379–1384, December 2013Google Scholar
  17. 17.
    Guan, Q., Fu, S., DeBardeleben, N., Blanchard, S.: Exploring time and frequency domains for accurate and automated anomaly detection in cloud computing systems. In: 2013 IEEE 19th Pacific Rim International Symposium on Dependable Computing (PRDC), pp. 196–205. IEEE (2013)Google Scholar
  18. 18.
    Gul, I., Hussain, M.: Distributed cloud intrusion detection model. Int. J. Adv. Sci. Technol. 34, 71–82 (2011)Google Scholar
  19. 19.
    Hodge, V.J., Austin, J.: A survey of outlier detection methodologies. Artif. Intell. Rev. 22(2), 85–126 (2004)CrossRefzbMATHGoogle Scholar
  20. 20.
    Ibidunmoye, O., Hernández-Rodriguez, F., Elmroth, E.: Performance anomaly detection and bottleneck identification. ACM Comput. Surv. 48(1), 1–35 (2015)CrossRefGoogle Scholar
  21. 21.
    Lin, M., Yao, Z., Gao, F., Li, Y.: Toward anomaly detection in IaaS cloud computing platforms. Int. J. Secur. Appl. 9(12), 175–188 (2015)Google Scholar
  22. 22.
    Liu, A., Chen, J.X., Wechsler, H.: Real-time timing channel detection in an software-defined networking virtual environment. Intell. Inf. Manag. 7(06), 283 (2015)Google Scholar
  23. 23.
    Mi, H., Wang, H., Yin, G., Cai, H., Zhou, Q., Sun, T., Zhou, Y.: Magnifier: online detection of performance problems in large-scale cloud computing systems. In: 2011 IEEE International Conference on Services Computing (SCC), pp. 418–425, July 2011Google Scholar
  24. 24.
    Penn, B.S.: Using self-organizing maps to visualize high-dimensional data. Comput. Geosci. 31(5), 531–544 (2005)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Reynolds, P., Killian, C., Wiener, J.L., Mogul, J.C., Shah, M.A., Vahdat, A.: PIP: detecting the unexpected in distributed systems. In: Proceedings of the 3rd Conference on Networked Systems Design and Implementation, NSDI 2006, Berkeley, CA, USA, vol. 3. USENIX Association (2006)Google Scholar
  26. 26.
    Sha, W., Zhu, Y., Chen, M., Huang, T.: Statistical learning for anomaly detection in cloud server systems: a multi-order Markov chain framework. IEEE Trans. Cloud Comput. (2015). https://doi.org/10.1109/TCC.2015.2415813
  27. 27.
    Song, X., Wu, M., Jermaine, C., Ranka, S.: Conditional anomaly detection. IEEE Trans. Knowl. Data Eng. 19(5), 631–645 (2007)CrossRefGoogle Scholar
  28. 28.
    Videla, A., Williams, J.J.W.: RabbitMQ in Action: Distributed Messaging for Everyone. Manning Publications Company, Grand Forks (2012)Google Scholar
  29. 29.
    Wang, C., Talwar, V., Schwan, K., Ranganathan, P.: Online detection of utility cloud anomalies using metric distributions. In: 2010 IEEE Network Operations and Management Symposium (NOMS), pp. 96–103, April 2010Google Scholar
  30. 30.
    Wang, C., Viswanathan, K., Choudur, L., Talwar, V., Satterfield, W., Schwan, K.: Statistical techniques for online anomaly detection in data centers. In: 2011 IFIP/IEEE International Symposium on Integrated Network Management (IM), pp. 385–392, May 2011Google Scholar
  31. 31.
    Zhang, Z., Wang, Y., Wang, K.: Fault diagnosis and prognosis using wavelet packet decomposition, Fourier transform and artificial neural network. J. Intell. Manuf. 24(6), 1213–1227 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • David O’Shea
    • 1
  • Vincent C. Emeakaroha
    • 1
    Email author
  • Neil Cafferkey
    • 1
  • John P. Morrison
    • 1
  • Theo Lynn
    • 2
  1. 1.Irish Centre for Cloud Computing and CommerceUniversity College CorkCorkIreland
  2. 2.Irish Centre for Cloud Computing and CommerceDublin City UniversityDublin 9Ireland

Personalised recommendations