Advertisement

Detecting Malicious URLs Using a Deep Learning Approach Based on Stacked Denoising Autoencoder

  • Huaizhi YanEmail author
  • Xin Zhang
  • Jiangwei Xie
  • Changzhen Hu
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 960)

Abstract

As the source of spamming, phishing, malware and many more such attacks, malicious URL is a chronic and complicated problem on the Internet. Machine learning approaches have taken effect and obtained high accuracy in detecting malicious URL. But the tedious process of extracting features from URL and the high dimension of feature vector makes the implementing time consuming. This paper presents a deep learning method using Stacked denoising autoencoders model to learn and detect intrinsic malicious features. We employ an SdA network to analyze URLs and extract features automatically. Then a logistic regression is implemented to detect malicious and benign URLs, which can generate detection models without a manually feature engineering. We have implemented our network model using Keras, a high-level neural networks API with a Tensor-flow backend, an open source deep learning library. 5 datasets were used and 4 other method were compared with our model. In the result, our architecture achieves an accuracy of 98.25% and a micro-averaged F1 score of 0.98, tested on a mixed dataset containing around 2 million samples.

Keywords

Network security Malicious URL detection Deep learning Stacked denoising autoencoder 

Notes

Acknowledgments

This work was supported by the National Key R&D Program of China (Grant No. 2016YFB0800700 & No. 2016YFC1000300).

References

  1. 1.
    Ma, J., Saul, L.K., Savage, S., Voelker, G.M.: Beyond blacklists: learning to detect malicious web sites from suspicious URLs. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1245–1254. ACM (2009)Google Scholar
  2. 2.
    Canali, D., Cova, M., Vigna, G., Kruegel, C.: Prophiler: a fast filter for the large-scale detection of malicious web pages. In: Proceedings of the 20th International Conference on World Wide Web, pp. 197–206. ACM (2011)Google Scholar
  3. 3.
    Wang, D., Navathe, S.B., Liu, L., Irani, D., Tamersoy, A., Pu, C.: Click traffic analysis of short URL spam on twitter. In: 2013 9th International Conference on Collaborative Computing: Networking, Applications and Worksharing (Collaboratecom), pp. 250–259. IEEE (2013)Google Scholar
  4. 4.
    Eshete, B., Villafiorita, A., Weldemariam, K.: BINSPECT: holistic analysis and detection of malicious web pages. In: SecureComm, pp. 149–166 (2012)Google Scholar
  5. 5.
    Berners-Lee, T., Masinter, L., McCahill, M.: Uniform resource locators (URL). Technical report (1994)Google Scholar
  6. 6.
    Zhang, H.-L., Zou, W., Han, X.-H.: Drive-by-download mechanisms and defenses. J. Softw. 24(4), 843–858 (2013). (in Chinese)CrossRefGoogle Scholar
  7. 7.
    Sha, H.-Z., Zhou, Z., Liu, Q.-Y., Qin, P.: Light-weight self-learning for URL classification. J. Commun. 35(9), 32–39 (2014)Google Scholar
  8. 8.
    Klien, F., Strohmaier, M.: Short links under attack: geographical analysis of spam in a URL shortener network. In: Proceedings of the 23rd ACM Conference on Hypertext and Social Media, pp. 83–88. ACM (2012)Google Scholar
  9. 9.
    Seifert, C., Welch, I., Komisarczuk, P.: Identification of malicious web pages with static heuristics. In: 2008 Australasian Telecommunication Networks and Applications Conference, ATNAC 2008, pp. 91–96. IEEE (2008)Google Scholar
  10. 10.
    Chollet, F.: Keras (2015). https://github.com/fchollet/keras
  11. 11.
    Abadi, M., et al.: Tensorflow: large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016)
  12. 12.
    Mohammad, R.M., Thabtah, F., McCluskey, L.: Predicting phishing websites based on self-structuring neural network. Neural Comput. Appl. 25(2), 443–458 (2014)CrossRefGoogle Scholar
  13. 13.
    Woodbridge, J., Anderson, H.S., Ahuja, A., Grant, D.: Predicting domain generation algorithms with long short-term memory networks. arXiv preprint arXiv:1611.00791 (2016)
  14. 14.
    Wang, Y., Cai, W.D., Wei, P.C.: A deep learning approach for detecting malicious JavaScript code. Secur. Commun. Netw. 9(11), 1520–1534 (2016)CrossRefGoogle Scholar
  15. 15.
    Bahnsen, A.C., Bohorquez, E.C., Villegas, S., Vargas, J., González, F.A.: Classifying Phishing URLs Using Recurrent Neural Networks (2017)Google Scholar
  16. 16.
    Sha, H.-Z., Liu, Q.-Y., Liu, T.-W.: Survey on malicious webpage detection research. Chin. J. Comput. 39(3), 529–542 (2016)MathSciNetGoogle Scholar
  17. 17.
    Sahoo, D., Liu, C., Hoi, S.C.: Malicious URL detection using machine learning: a survey. arXiv preprint arXiv:1701.07179 (2017)
  18. 18.
    Wang, D., Navathe, S.B., Liu, L., Irani, D., Tamersoy, A., Pu, C.: Click traffic analysis of short URL spam on Twitter. In: 2013 9th International Conference Conference on Collaborative Computing: Networking, Applications and Worksharing (Collaboratecom), pp. 250–259. IEEE (2013)Google Scholar
  19. 19.
    Thomas, K., Grier, C., Ma, J., Paxson, V., Song, D.: Design and evaluation of a real-time URL spam filtering service. In: 2011 IEEE Symposium on Security and Privacy (SP), pp. 447–462. IEEE (2011)Google Scholar
  20. 20.
    Pao, H.K., Chou, Y.L., Lee, Y.J.: Malicious URL detection based on kolmogorov complexity estimation. In: Proceedings of the 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology, vol. 01, pp. 380–387. IEEE Computer Society (2012)Google Scholar
  21. 21.
    Yousefi-Azar, M., Varadharajan, V., Hamey, L., Tupakula, U.: Autoencoder-based feature learning for cyber security applications. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 3854–3861. IEEE (2012)Google Scholar
  22. 22.
    Does Alexa have a list of its top-ranked websites? https://support.alexa.com/hc/en-us/articles/200449834-Does-Alexa-have-a-list-of-its-top-ranked-websites. Accessed 06 Apr 2016
  23. 23.
    Zhao, P., Hoi, S.C.H.: Cost-sensitive online active learning with application to malicious URL detection. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 919–927. ACM (2013)Google Scholar
  24. 24.
    Le Roux, N., Bengio, Y.: Deep belief networks are compact universal approximators. Neural Comput. 22(8), 2192–2207 (2010)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Lipton, Z.C., Berkowitz, J., Elkan, C.: A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019 (2015)
  26. 26.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  27. 27.
    Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)MathSciNetzbMATHGoogle Scholar
  28. 28.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Hinton, G.: A practical guide to training restricted Boltzmann machines. Momentum 9(1), 926 (2010)Google Scholar
  30. 30.
    Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Vincent, P., Larochelle, H., Bengio, Y., et al.: Extracting and composing robust features with denoising autoencoders. In: International Conference on Machine Learning, pp. 1096–1103. ACM (2008)Google Scholar
  32. 32.
    Menon, A.K.: Large-Scale Support Vector Machines: Algorithms and Theory. Research Exam (2009)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Huaizhi Yan
    • 1
    • 2
    Email author
  • Xin Zhang
    • 1
    • 2
  • Jiangwei Xie
    • 1
    • 2
  • Changzhen Hu
    • 1
    • 2
  1. 1.School of ComputerBeijing Institute of TechnologyBeijingChina
  2. 2.Beijing Key Laboratory of Software Security Engineering Technology (Beijing Institute of Technology)BeijingChina

Personalised recommendations