Malware: From Modelling to Practical Detection

Abstract

Malicious Software referred to as Malware refers to a software that has infiltrated to a computer without the authorization of the computer (or the owner of the computer). Typical categories of malicious code include Trojan Horses, viruses, worms etc. Malware has been a major cause of concern for information security. With the growth in complexity of computing systems and the ubiquity of information due to WWW, detection of malware has become horrendously complex. In this paper, we shall survey the theory behind malware to provide the challenges behind detection of malware. It is of interest to note that the power of the malware (or for that matter computer warfare) can be seen in the theories proposed by the iconic scientists Alan Turing and John von Neumann. The malicious nature of malware can be broadly categorized as injury and infection analogously in the epidemiological framework. On the same lines, the remedies can also be thought of through analogies with epidemiological notions like disinfection, quarantine, environment control etc. We shall discuss these aspects and relate the above to notions of computability.

Adleman in his seminal paper has extrapolated protection mechanisms such as quarantine, disinfection and certification. It may be noted that most of the remedies in general are undecidable. We shall discuss remedies that are being used and contemplated. One of the well-known restricted kind of remedies is to search for signatures of possible malwares and detect them before getting it through to the computer. Large part of the current remedies rely on signature based approaches that is, heavy reliance on the detection of syntactic patterns. Recent trends in security incidence reports show a huge increase in obfuscated exploits; note that in the majority of obfuscators, the execution behaviour remains the same while it can escape syntactic recognitions. Further, malware writers are using a combination of features from various types of classic malwares such as viruses and worms. Thus, it has become all the more necessary to take a holistic approach and arrive at detection techniques that are based on characterizations of malware behaviour that includes the environment in which it is expected to execute.

In the paper, we shall first survey various approaches of behavioural characterization of malware, difficulties of virus detection, practical virus detection techniques and protection mechanisms from viruses. Towards the end of the paper, we shall briefly discuss our new approach of detecting malware via a new method of validation in a quarantine environment and show our preliminary results for the detection of malware on systems that are expected to carry a priori known set of software.

The work was partially supported under Indo-Trento Promotion for Advanced Research.