Skip to main content

A Three-Way Decision Making Approach to Malware Analysis

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9436))

Abstract

Malware analysis techniques generally classify software behaviors as malicious (i.e., harmful) or benign (i.e., not harmful). Due to ambiguous nature of application behavior, there are cases where it may not be possible to confidently reach two-way conclusions. This may result in higher classification errors which in turn affect users trust on malware analysis outcomes. In this paper, we investigate a three-way decision making approach based on probabilistic rough set models, such as, information-theoretic rough sets and game-theoretic rough sets, for malware analysis. The essential idea is to add a third option of deferment or delaying a decision whenever the available information is not sufficient to reach certain conclusions. We demonstrate the applicability of the proposed approach with an example from system call sequences of a vulnerable Linux application.

This is a preview of subscription content, log in via an institution.

References

  1. Azam, N., Yao, J.T.: Analyzing uncertainties of probabilistic rough set regions with game-theoretic rough sets. Int. J. Approx. Reasoning 55(1), 142–155 (2014)

    Article  MathSciNet  Google Scholar 

  2. Biddle, R., van Oorschot, P.C., Patrick, A.S., Sobey, J., Whalen, T.: Browser interfaces and extended validation ssl certificates: an empirical study. In: Proceedings of the 2009 ACM Workshop on Cloud Computing Security, pp. 19–30 (2009)

    Google Scholar 

  3. Demchenko, Y., Ngo, C., de Laat, C., Membrey, P., Gordijenko, D.: Big security for big data: addressing security challenges for the big data infrastructure. In: Jonker, W., Petković, M. (eds.) SDM 2013. LNCS, vol. 8425, pp. 76–94. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  4. Deng, X., Yao, Y.: An information-theoretic interpretation of thresholds in probabilistic rough sets. In: Li, T., Nguyen, H.S., Wang, G., Grzymala-Busse, J., Janicki, R., Hassanien, A.E., Yu, H. (eds.) RSKT 2012. LNCS, vol. 7414, pp. 369–378. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  5. Egele, M., Scholte, T., Kirda, E., Kruegel, C.: A survey on automated dynamic malware-analysis techniques and tools. ACM Comput. Surv. 44(2), 6 (2012)

    Article  Google Scholar 

  6. Forrest, S., Hofmeyr, S.A., Somayaji, A., Longstaff, T.A.: UNM dataset. http://www.cs.unm.edu/immsec/data-sets.htm. Accessed 6 April 2015

  7. Forrest, S., Hofmeyr, S.A., Somayaji, A., Longstaff, T.A.: A sense of self for unix processes. In: IEEE Symposium on Security and Privacy, pp. 120–128 (1996)

    Google Scholar 

  8. Gandotra, E., Bansal, D., Sofat, S.: Malware analysis and classification: a survey. J. Inf. Secur. 5, 56–64 (2014)

    Google Scholar 

  9. Herbert, J.P., Yao, J.T.: Game-theoretic rough sets. Fundamenta Informaticae 108(3–4), 267–286 (2011)

    MathSciNet  MATH  Google Scholar 

  10. Leyton-Brown, K., Shoham, Y.: Essentials of Game Theory: A Concise Multidisciplinary Introduction. Morgan & Claypool Publishers, San Rafael (2008)

    MATH  Google Scholar 

  11. Liu, D., Yao, Y.Y., Li, T.R.: Three-way investment decisions with decision-theoretic rough sets. Int. J. Comput. Intel. Syst. 4(1), 66–74 (2011)

    Article  Google Scholar 

  12. Mehdi, B., Ahmed, F., Khayyam, S.A., Farooq, M.: Towards a theory of generalizing system call representation for in-execution malware detection. In: IEEE International Conference on Communications (ICC), pp. 1–5 (2010)

    Google Scholar 

  13. Pawlak, Z.: Rough sets. Int. J. Comput. Inf. Sci. 11, 241–256 (1982)

    Article  Google Scholar 

  14. Symantec: Internet security threat report, vol. 19. http://www.symantec.com/security_response/publications/threatreport.jsp. Accessed 12 April 2015

  15. Tanenbaum, A.S., Woodhull, A.S.: Operating systems: design and implementation, vol. 2. Prentice-Hall, Englewood Cliffs (1987)

    Google Scholar 

  16. Wright, C., Cowan, C., Morris, J., Smalley, S., Kroah-Hartman, G.: Linux security module framework. In: Ottawa Linux Symposium, vol. 8032, pp. 6–16 (2002)

    Google Scholar 

  17. Yang, H., Li, T., Hu, X., Wang, F., Zou, Y.: A survey of artificial immune system based intrusion detection. The Scientific World Journal (2014)

    Google Scholar 

  18. Yao, Y.Y.: Probabilistic rough set approximations. Int. J. Approx. Reasoning 49(2), 255–271 (2008)

    Article  Google Scholar 

  19. Yao, Y.Y.: Three-way decisions with probabilistic rough sets. Inf. Sci. 180(3), 341–353 (2010)

    Article  MathSciNet  Google Scholar 

  20. Yao, Y.Y.: Two semantic issues in a probabilistic rough set model. Fundamenta Informaticae 108(3–4), 249–265 (2011)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgment

This work was partially supported by a discovery grant from NSERC canada.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nouman Azam .

Editor information

Editors and Affiliations

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (http://creativecommons.org/licenses/by-nc/2.5/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Nauman, M., Azam, N., Yao, J. (2015). A Three-Way Decision Making Approach to Malware Analysis. In: Ciucci, D., Wang, G., Mitra, S., Wu, WZ. (eds) Rough Sets and Knowledge Technology. RSKT 2015. Lecture Notes in Computer Science(), vol 9436. Springer, Cham. https://doi.org/10.1007/978-3-319-25754-9_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25754-9_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25753-2

  • Online ISBN: 978-3-319-25754-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics