Abstract
The inability of existing architectures to allow corporations to quickly process information at scale and share knowledge with peers makes it difficult for malware analysis researchers to present a clear picture of criminal activity. Hence, analysis is limited in effectively and accurately identify the full scale of adversaries’ activities and develop effective mitigation strategies. In this paper, we present Skald: a novel architecture which guides the creation of analysis systems to support the research of malicious activities plaguing computer systems. Our design provides the scalability, flexibility, and robustness needed to process current and future volumes of data. We show that our prototype is able to process millions of samples in only few milliseconds per sample with zero critical errors. Additionally, Skald enables the development of new methodologies for information sharing, enabling analysis across collective knowledge. Consequently, defenders can perform accurate investigations and real-time discovery, while reducing mitigation time and infrastructure cost.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Alvarez, V.M.: Yara 3.3.0. VirusTotal (Google Inc.) (2015). http://plusvic.github.io/yara/
Barack, O.: Executive Order No. 13691. Promoting Private Sector Cybersecurity Information Sharing (2015)
Bu, Z., Dirro, T., Greve, P., Lin, Y., Marcus, D., Paget, F., Pogulievsky, V., Schmugar, C., Shah, J., Sommer, D., et al.: McAfee Threats Report: Second Quarter 2012 (2012)
Choo, K.-K.R.: The cyber threat landscape: challenges and future research directions. Comput. Secur. 30(8), 719–731 (2011)
Cristian, F.: Understanding fault-tolerant distributed systems. Commun. ACM 34(2), 56–78 (1991)
DARPA: Cyber Information Sharing - DARPA Cyber Forum, October 2015
Estublier, J.: Software configuration management: a roadmap. In: Conference on the Future of Software Engineering (2000)
Google: Protocol Buffers, November 2015. https://developers.google.com/protocol-buffers/
Grobauer, B., Berger, S., Göbel, J., Schreck, T., Wallinger, J.: The MANTIS Framework: Cyber Threat Intelligence Management for CERTs, Boston, USA, June 2014
Guarnieri, C., Tanasi, A., Bremer, J., Schloesser, M.: The Cuckoo Sandbox (2012). http://cuckoosandbox.org
Hanif, Z., Calhoun, T., Trost, J.: BinaryPig: scalable static binary analysis over Hadoop. In: Black Hat USA (2013)
HiveMQ: MQTT Essentials Part 6: Quality of Service 0, 1 & 2 (2015). http://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels
Jang, J., Brumley, D., Venkataraman, S.: BitShred: feature hashing malware for scalable triage and semantic analysis. In: Conference on Computer and Communications Security, CCS (2011)
Kolosnjaji, B., Zarras, A., Lengyel, T., Webster, G., Eckert, C.: Adaptive semantics-aware malware classification. In: Caballero, J., Zurutuza, U., Rodríguez, R.J. (eds.) DIMVA 2016. LNCS, vol. 9721, pp. 419–439. Springer, Heidelberg (2016). doi:10.1007/978-3-319-40667-1_21
Krafzig, D., Banke, K., Slama, D.: Enterprise SOA: Service-Oriented Architecture Best Practices. Prentice Hall Professional, Indianapolis (2005)
Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. ACM SIGOPS Oper. Syst. Rev. 44(2), 35–40 (2010)
Lengyel, T.K., Maresca, S., Payne, B.D., Webster, G.D., Vogl, S., Kiayias, A.: Scalability, fidelity and stealth in the DRAKVUF dynamic malware analysis system. In: Annual Computer Security Applications Conference, ACSAC (2014)
Ollmann, G.: Behind todays crimeware installation lifecycle: how advanced malware morphs to remain stealthy and persistent. Technical report, Damballa (2011)
Papazoglou, M.P., Van Den Heuvel, W.-J.: Service oriented architectures: approaches, technologies and research issues. VLDB J. 16(3), 389–415 (2007)
Parkour, M., DiMino, A.: Deepend Research - Yara Exchange, May 2015. http://www.deependresearch.org/2012/08/yara-signature-exchange-google-group.htm
Shields, W.: Problems with PEHash Implementations, September 2014. https://gist.github.com/wxsBSD/07a5709fdcb59d346e9e
Stamos, A.: The Failure of the Security Industry, April 2015. http://www.scmagazine.com/the-failure-of-the-security-industry/article/403261/
The MITRE Corporation: Collaborative Research Into Threats (CRITs), June 2014. http://www.mitre.org/capabilities/cybersecurity/overview/cybersecurity-blog/collaborative-research-into-threats-crits
Verma, A., Pedrosa, L., Korupolu, M.R., Oppenheimer, D., Tune, E., Wilkes, J.: Large-scale cluster management at Google with Borg. In: European Conference on Computer Systems, EuroSys (2015)
VirusTotal: File Statistics, May 2015. https://www.virustotal.com/en/statistics/
Vixie, P.: Internet Security Marketing: Buyer Beware, April 2015. http://www.circleid.com/posts/20150420_internet_security_marketing_buyer_beware/
Wicherski, G.: PEHash: a novel approach to fast malware clustering. In: USENIX Workshop on Large-Scale Exploits and Emergent Threats, LEET (2009)
Zeltser, L.: SANS - Managing and Exploring Malware Samples with Viper, June 2014. https://digital-forensics.sans.org/blog/2014/06/04/managing-and-exploring-malware-samples-with-viper
Acknowledgments
We would like to thank the Technical University of Munich for providing ample infrastructure to support our prototype development. We would also like to thank the United States Air Force for sponsoring George Webster in his academic pursuit. In addition, we thank the German Federal Ministry of Education and Research for providing funding for hardware under grant 16KIS0328 (IUNO). Lastly, we would like to thank the members of VirusTotal, Yara Exchange, and DARPA for their valuable discussions and support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Webster, G.D., Hanif, Z.D., Ludwig, A.L.P., Lengyel, T.K., Zarras, A., Eckert, C. (2016). SKALD: A Scalable Architecture for Feature Extraction, Multi-user Analysis, and Real-Time Information Sharing. In: Bishop, M., Nascimento, A. (eds) Information Security. ISC 2016. Lecture Notes in Computer Science(), vol 9866. Springer, Cham. https://doi.org/10.1007/978-3-319-45871-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-45871-7_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-45870-0
Online ISBN: 978-3-319-45871-7
eBook Packages: Computer ScienceComputer Science (R0)