Abstract
Feature extraction and feature selection are the first tasks in pre-processing of input logs in order to detect cybersecurity threats and attacks by utilizing data mining techniques in the field of Artificial Intelligence. When it comes to the analysis of heterogeneous data derived from different sources, these tasks are found to be time-consuming and difficult to be managed efficiently.
In this paper, we present an approach for handling feature extraction and feature selection utilizing machine learning algorithms for security analytics of heterogeneous data derived from different network sensors. The approach is implemented in Apache Spark, using its python API, named pyspark.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python. O’Reilly Media Inc., Sebastopol (2009)
Veeramachaneni, K., Arnaldo, I., Cuesta-Infante, A., Korrapati, V., Bassias, C., Li, K.: AI2: training a big data machine to defend. In: IEEE International Conference on Big Data Security, New York, NY, USA, June 2016
Shyu, M.-L., Huang, Z., Luo, H.: Efficient mining and detection of sequential intrusion patterns for network intrusion detection systems. In: Yu, P.S., Tsai, J.J.P. (eds.) Machine Learning in Cyber Trust, pp. 133–154. Springer, Heidelberg (2009). https://doi.org/10.1007/978-0-387-88735-7_6
Sisiaridis, D., Carcillo, F., Markowitch, O.: A framework for threat detection in communication systems. In: Proceedings of the 20th Pan-Hellenic Conference on Informatics, pp. 68:1–68:6. ACM (2016)
Sisiaridis, D., Kuchta, V., Markowitch, O.: A categorical approach in handling event-ordering in distributed systems. In: Parallel and Distributed Systems (ICPADS), pp. 1145–1150. IEEE (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Sisiaridis, D., Markowitch, O. (2018). Automating Feature Extraction and Feature Selection in Big Data Security Analytics. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2018. Lecture Notes in Computer Science(), vol 10842. Springer, Cham. https://doi.org/10.1007/978-3-319-91262-2_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-91262-2_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91261-5
Online ISBN: 978-3-319-91262-2
eBook Packages: Computer ScienceComputer Science (R0)