Advertisement

RAT Hunter: Building Robust Models for Detecting Remote Access Trojans Based on Optimum Hybrid Features

  • Mohammad Mehdi BehradFar
  • Hamed HaddadPajouhEmail author
  • Ali Dehghantanha
  • Amin Azmoodeh
  • Hadis Karimipour
  • Reza M. Parizi
  • Gautam Srivastava
Chapter

Abstract

Nowadays, critical infrastructures are severely exposed to a wide range of malicious attempts. The malicious activities are becoming more sophisticated. They infect victim’s machines and seek to obtain information from users instead of doing a destructiveness to the machine. Remote Access Trojan (RAT) is a type of malware that tries to control the victim’s machine remotely without victim awareness. Accordingly, the number and harmful effect of RAT threats for information thieves has increased dramatically. In this chapter, we propose an optimum feature set for hunting RAT malware based on intelligence feature selection for machine learning classification tasks. For building a robust model, we collected real-world samples from well-known repositories like Virus Total and Virus Share. Afterwards, the behaviour of these types of malware are analyzed through a modified sandbox as a reverse engineering tool to extract features from dynamic and static analysis. With the feature selection process, we applied a two-layer feature selection algorithm like information gain and correlated feature selection for obtaining the optimum set of features to tackles RAT threats. By implementing different models like the generative and deep learning models, we obtained an accuracy rate of 99.75% and a false alarm rate of 0.3%.

References

  1. 1.
    E.M. Dovom, A. Azmoodeh, A. Dehghantanha, D.E. Newton, R.M. Parizi, H. Karimipour, Fuzzy pattern tree for edge malware detection and categorization in IoT. J. Syst. Archit. 97, 1–7 (2019). https://doi.org/10.1016/j.sysarc.2019.01.017 CrossRefGoogle Scholar
  2. 2.
    J. Sakhnini, H. Karimipour, A. Dehghantanha, R.M. Parizi, G. Srivastava, Security aspects of internet of things aided smart grids: a bibliometric survey. Internet of Things 2019, 100111 (2019). https://doi.org/10.1016/j.iot.2019.100111 CrossRefGoogle Scholar
  3. 3.
    R.T. Shoniwa, G. George, Scanning tool for the detection of images embedded with malicious programs, in 2015 International Conference on Electrical, Electronics, Signals, Communication and Optimization (EESCO) (2015)Google Scholar
  4. 4.
    D. Kiwia, A. Dehghantanha, K.-K.R. Choo, J. Slaughter, A cyber kill chain based taxonomy of banking Trojans for evolutionary computational intelligence. J. Comput. Sci. 27, 394–409 (2018)CrossRefGoogle Scholar
  5. 5.
    P.N. Bahrami, A. Dehghantanha, T. Dargahi, R.M. Parizi, K.R. Choo, H.H.S. Javadi, Cyber kill chain-based taxonomy of advanced persistent threat actors: analogy of tactics, techniques, and procedures. J. Inf. Process. Syst. 15, 865–889 (2019).  https://doi.org/10.3745/JIPS.03.0126 Google Scholar
  6. 6.
    S.C. Pallaprolu, J.M. Namayanja, V.P. Janeja, C.S. Adithya, Label propagation in big data to detect remote access Trojans, in 2016 IEEE International Conference on Big Data (Big Data) (IEEE, Piscataway, 2016), pp. 3539–3547CrossRefGoogle Scholar
  7. 7.
    R. HosseiniNejad, H. HaddadPajouh, A. Dehghantanha, R. M. Parizi, A cyber kill chain based analysis of remote access Trojans, in Handbook of Big Data and IoT Security, ed. by A. Dehghantanha, K.-K.R. Choo (Springer, Cham, 2019), pp. 273–299. https://doi.org/10.1007/978-3-030-10543-3_12 CrossRefGoogle Scholar
  8. 8.
    T. Dargahi, A. Dehghantanha, P.N. Bahrami, M. Conti, G. Bianchi, L. Benedetto, A Cyber-Kill-Chain based taxonomy of crypto-ransomware features. J. Comput. Virol. Hack Tech. 15(4), 277–305 (2019). https://doi.org/10.1007/s11416-019-00338-7 CrossRefGoogle Scholar
  9. 9.
    S. Samuel, J. Graham, C. Hinds, Hunting Malware: An example using Gh0st, in 2017 International Conference on Computational Science and Computational Intelligence (CSCI) (IEEE, 2017 Dec), pp. 97–102Google Scholar
  10. 10.
    H. Mwiki, T. Dargahi, A. Dehghantanha, K.-K.R. Choo, Analysis and triage of advanced hacking groups targeting western countries critical national infrastructure: APT28, RED October, and Regin, in Critical Infrastructure Security and Resilience (Springer, Berlin, 2019), pp. 221–244Google Scholar
  11. 11.
    M. Rezaeirad, B. Farinholt, H. Dharmdasani, P. Pearce, K. Levchenko, D. McCoy, Schrödinger’s RAT: profiling the stakeholders in the remote access Trojan ecosystem, in 27th USENIX Security Symposium (USENIX Security 18) (2018), pp. 1043–1060Google Scholar
  12. 12.
    M. Mimura, Y. Otsubo, H. Tanaka, Evaluation of a brute forcing tool that extracts the rat from a malicious document file, in 2016 11th Asia Joint Conference on Information Security (AsiaJCIS) (IEEE, Piscataway, 2016), pp. 147–154Google Scholar
  13. 13.
    A. Pektaş, T. Acarman, Classification of malware families based on runtime behaviors. J. Inform. Secur. Appl. 37, 91–100 (2017)Google Scholar
  14. 14.
    S. Wu, S. Liu, W. Lin, X. Zhao, S. Chen, Detecting remote access Trojans through external control at area network borders, in Proceedings of the Symposium on Architectures for Networking and Communications Systems (IEEE Press, New York, 2017), pp. 131–141Google Scholar
  15. 15.
    H.H. Pajouh, G. Dastghaibyfard, S. Hashemi, Two-tier network anomaly detection model: a machine learning approach. J. Intell. Inf. Syst. 48(1), 61–74 (2017). https://doi.org/10.1007/s10844-015-0388-x CrossRefGoogle Scholar
  16. 16.
    R.M. Parizi, A. Dehghantanha, K.-K.R. Choo, A. Singh, Empirical vulnerability analysis of automated smart contracts security testing on blockchains, in Proceedings of the 28th Annual International Conference on Computer Science and Software Engineering, CASCON ’18 (2018), pp. 103–113Google Scholar
  17. 17.
    D. Jiang, K. Omote, An approach to detect remote access Trojan in the early stage of communication, in 2015 IEEE 29th International Conference on Advanced Information Networking and Applications (IEEE, Piscataway, 2015), pp. 706–713Google Scholar
  18. 18.
    M. Yamada, M. Morinaga, Y. Unno, S. Torii, M. Takenaka, RAT-based malicious activities detection on enterprise internal networks, in 2015 10th International Conference for Internet Technology and Secured Transactions (ICITST) (IEEE, Piscataway, 2015), pp. 321–325Google Scholar
  19. 19.
    A.A. Awad, S.G. Sayed, S.A. Salem, A network-based framework for RAT-bots detection, in 2017 8th IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON) (IEEE, Piscataway, 2017), pp. 128–133Google Scholar
  20. 20.
    B. Kolosnjaji, A. Zarras, G. Webster, C. Eckert, Deep learning for classification of malware system call sequences, in Australasian Joint Conference on Artificial Intelligence (Springer, 2016), pp. 137–149Google Scholar
  21. 21.
    H. HaddadPajouh, A. Dehghantanha, R. Khayami, K.-K.R. Choo, A deep Recurrent Neural Network based approach for Internet of Things malware threat hunting. Futur. Gener. Comput. Syst. 85, 88–96 (2018)CrossRefGoogle Scholar
  22. 22.
    P. Wang, Y.-S. Wang, Malware behavioural detection and vaccine development by using a support vector model classifier. J. Comput. Syst. Sci. 81(6), 1012–1026 (2015)CrossRefGoogle Scholar
  23. 23.
    Z. Xu, S. Ray, P. Subramanyan, S. Malik, Malware detection using machine learning based analysis of virtual memory access patterns, in Proceedings of the Conference on Design, Automation and Test in Europe, European Design and Automation Association (2017), pp. 169–174Google Scholar
  24. 24.
    M. Sikorski, A. Honig, Practical Malware Analysis: The Hands-on Guide to Dissecting Malicious Software (No Starch Press, San Francisco, 2012)Google Scholar
  25. 25.
    J.M. Van Campenhout, On the peaking of the Hughes mean recognition accuracy: The resolution of an apparent paradox. IEEE Trans. Syst. Man Cybern. 8(5), 390–395 (1978 May)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Y. Yang, J.O. Pedersen, A comparative study on feature selection in text categorization, in Proceedings of the International Conference on Machine Learning, vol. 97 (1997), p. 35Google Scholar
  27. 27.
    M.A. Hall, Correlation-based feature selection for machine learning. Ph.D Thesis, The University of Waikato, Hamilton, 1999Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Computer Engineering DepartmentPishtazan Institute of Higher EducationShirazIran
  2. 2.Cyber Science LabUniversity of GuelphGuelphCanada
  3. 3.School of EngineeringUniversity of GuelphGuelphCanada
  4. 4.College of Computer and Software EngineeringKennesaw State UniversityMariettaUSA
  5. 5.Department of Mathematics and Computer ScienceBrandon UniversityBrandonCanada

Personalised recommendations