Feature Based Techniques for Auto-Detection of Novel Email Worms

  • Mohammad M. Masud
  • Latifur Khan
  • Bhavani Thuraisingham
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4426)


This work focuses on applying data mining techniques to detect email worms. We apply a feature-based detection technique. These features are extracted using different statistical and behavioral analysis of emails sent over a certain period of time. The number of features thus extracted is too large. So, our goal is to select the best set of features that can efficiently distinguish between normal and viral emails using classification techniques. First, we apply Principal Component Analysis (PCA) to reduce the high dimensionality of data and to find a projected, optimal set of attributes. We observe that the application of PCA on a benchmark dataset improves the accuracy of detecting novel worms. Second, we apply J48 decision tree algorithm to determine the relative importance of features based on information gain. We are able to identify a subset of features, along with a set of classification rules that have a better performance in detecting novel worms than the original set of features or PCA-reduced features. Finally, we compare our results with published results and discuss our future plans to extend this work.


Email worm data mining feature selection Principal Component Analysis classification technique 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Martin, S., et al.: A Two-Layer Approach for Novel Email Worm Detection. Submitted to USENIX SRUTI (Steps on Reducing Unwanted Traffic on the Internet) (2005)Google Scholar
  2. 2.
    WEKA: Data Mining Software in Java.
  3. 3.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
  4. 4.
    Golbeck, J., Hendler, J.: Reputation network analysis for email filtering. In: CEAS (2004)Google Scholar
  5. 5.
    Newman, M.E.J., Forrest, S., Balthrop, J.: Email networks and the spread of computer viruses. Physical Review E 66, 35101 (2002)CrossRefGoogle Scholar
  6. 6.
    Symantec Corporation. W32.Beagle.BG. Online (2005), Scholar
  7. 7.
    Schultz, M., Eskin, E., Zadok, E.: MEF: Malicious email filter, a UNIX mail filter that detects malicious windows executables. In: USENIX Annual Technical Conference - FREENIX Track (June 2001)Google Scholar
  8. 8.
    Singh, S., et al.: The EarlyBird System for Real-time Detection of Unknown Worms. Technical report - cs2003-0761, UCSD (2003)Google Scholar
  9. 9.
    Kim, H.-A., Karp, B.: Autograph: Toward Automated, Distributed Worm Signature Detection. In: Proceedings of the 13th Usenix Security Symposium (Security 2004), San Diego, CA (August 2004)Google Scholar
  10. 10.
    Newsome, J., Karp, B., Song, D.: Polygraph: Automatically Generating Signatures for Polymorphic Worms. In: Proceedings of the IEEE Symposium on Security and Privacy, May 2005, IEEE Computer Society Press, Los Alamitos (2005)Google Scholar
  11. 11.
  12. 12.
    Martin, S., et al.: Analyzing Behavioral Features for Email Classification. In: Proceedings of the IEEE Second Conference on Email and Anti-Spam (CEAS 2005), Stanford University, July 21-22, 2005, IEEE Computer Society Press, Los Alamitos (2005)Google Scholar
  13. 13.
    Sidiroglou, S., et al.: An Email Worm Vaccine Architecture. In: Deng, R.H., et al. (eds.) ISPEC 2005. LNCS, vol. 3439, Springer, Heidelberg (2005)Google Scholar
  14. 14.
    Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)zbMATHGoogle Scholar
  15. 15.
    John, G.H., Langley, P.: Estimating Continuous Distributions in Bayesian Classifiers. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 338–345. Morgan Kaufmann, San Mateo (1995)Google Scholar
  16. 16.
    Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Haussler, D. (ed.) 5th Annual ACM Workshop on COLT, Pittsburgh, PA, pp. 144–152. ACM Press, New York (1992)Google Scholar
  17. 17.
    A library for Support Vector Machine:

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Mohammad M. Masud
    • 1
  • Latifur Khan
    • 1
  • Bhavani Thuraisingham
    • 1
  1. 1.Department of Computer Science, The University of Texas at Dallas, Richardson, Texas-75083 

Personalised recommendations