Abstract
As the IP traffic observed on network operator’s backbones keeps increasing year by year, the analysis of NetFlow data metered for this traffic becomes a burden for centralized traffic monitoring solutions. Thus, SCRIPT proposes a decentralized accounting architecture and framework for NetFlow storage and analysis, which is flexible to allow for the development of distributed traffic analysis applications. SCRIPT mechanisms organize multiple PCs or AXP (Application Extension Platform) cards in an analysis network and route NetFlow records according to rules imposed by the analysis application. In turn, the evaluation of the prototype has shown that (a) this approach allows for a linear increase of the number of NetFlow records, which can be processed with the number of nodes in the SCRIPT deployment network, and (b) deploying SCRIPT on router-embedded AXP cards is improving an already existing infrastructure with the capability of storage and processing of NetFlow records.
C. Morariu and P. Racz have been with CSG@IFI at the time of performing this work. At the time of writing, Cristian had moved to Qnective AG and Peter to Ascom Schweiz AG.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bailey MD, Cooke E, Jahanian F, Nazario J (2005) The Internet motion sensor: A distributed blackhole monitoring system. In: 12th annual network and distributed system security symposium (NDSS’05), San Diego, Feb 2005
Brauckhoff D, Tellenbach B, Wagner A, May M, Lakhina A (2006) Impact of packet sampling on anomaly detection metrics. In: 6th ACM SIGCOMM Conference on Internet Measurements, Rio de Janeiro, Brazil, 17–25 Oct 2006
Claise B (ed) (2004) Cisco systems NetFlow services export version 9; Internet engineering task force, Internet engineering task force RFC 3954, Oct 2004
Claise B (ed) (2008) Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information; Internet Engineering Task Force RFC 5101, Jan 2008
Duffield N, Lund C, Thorup M (2001) Charging from sampled network usage. In: 1st ACM SIGCOMM Workshop on Internet Measurements, San Francisco, Nov 2001
FIPS 180-2 (2002) Secure Hash Standard (SHS), National Institute of Standards and Technology, Aug 2002, amended Feb 2004
Han SH, Kim MS, Ju HT, Hong JWK (2002) The architecture of NG-MON: a passive network monitoring system for high-speed IP networks. In 13th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management (DSOM’02), Montreal, Canada, Oct 2002
Henke C, Schmoll C, Zseby T (2008) Empirical evaluation of hash functions for multipoint measurements. ACM Computer Communication Review 38(3): 39–50
Jimenez R, Osmani F, Knutsson B (2011) Sub-second lookups on a large-scale Kademlia based overlay. In: 11th IEEE International Conference on Peer-to-Peer Computing 2011, Kyoto, Japan, Aug 2011
Kitatsuji Y, Yamazaki K (2004) A distributed real-time tool for IP-flow measurement. In: international symposium on applications and the Internet, Tokyo, Japan, Jan 2004
Maymounkov P, Mazières D (2002) Kademlia: a Peer-to-Peer information system based on the XOR metric. IPTPS, Cambridge
Mao Y, Chen K, Wang D, Zheng W (2001) Cluster-based online monitoring system of web traffic. In: 3rd International Workshop on Web Information and Data Management, Atlanta, Georgia, USA, Nov 2001
Morariu C, Racz P, Stiller B (2009) Design and implementation of a distributed platform for sharing IP flow records. In: 20th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management (DSOM’09), Venice, Italy, Oct 2009
Morariu C, Kramis T, Stiller B (2008) DIPStorage: distributed storage of IP flow records. In: 16th IEEE workshop on local and metropolitan area networks, Cluj-Napoca Romania, Sept 2008
Morariu C, Racz P, Stiller B (2010) SCRIPT: a framework for scalable real-time IP flow record analysis. In: 12th IEEE/IFIP Network Operations and Management Symposium (NOMS 2010), IEEE, Osaka, Japan, April 2010
Postel J (1980) User datagram protocol. Internet Engineering Task Force, RFC 768, August 1980
Rivest R (1992) The MD5 message-digest algorithm. Internet Engineering Task Force RFC 1321, April 1992
Schulzrinne H, Casner S, Frederick R, Jacobson V (2003) RTP: a transport protocol for real-time applications. Internet Engineering Task Force RFC 3550, July 2003
Stewart R, Xie Q, Morneault K, Sharp C, Schwarzbauer H, Taylor T, Rytina I, Kalla M, Zhang L, Paxson V (2000) Stream control transmission protocol. Internet Engineering Task Force RFC 2960, Oct 2000
Wikipedia (2011) NetFlow. http://en.wikipedia.org/wiki/Netflow
Zseby T, Boschi E, Brownlee N, Claise B (2007) IPFIX applicability. Internet Engineering Task Force, Internet Draft, http://www.ietf.org/internet-drafts/draft-ietf-ipfix-as-12.txt
Acknowledgements
This work was supported in part by the Cisco University Research Program Fund, Grant No. 2008-02735, in part by the DaSAHIT project funded by the Swiss National Science Foundations, Contract No. 200021-118128/1, and the IST Network of Excellence EMANICS funded by the European Union, Contract No. FP6-2004-IST-026854-NoE. The authors would like to express many thanks to Ralf Wolter, Benoit Claise, and David Hausheer for their valuable support and inspiring discussions as well as Alexander Clemm for his detailed feedback, which helped to improve this chapter.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media New York
About this chapter
Cite this chapter
Stiller, B., Morariu, C., Racz, P. (2013). Scalable and Robust Decentralized IP Traffic Flow Collection and Analysis (SCRIPT). In: Clemm, A., Wolter, R. (eds) Network-Embedded Management and Applications. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-6769-5_15
Download citation
DOI: https://doi.org/10.1007/978-1-4419-6769-5_15
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-6768-8
Online ISBN: 978-1-4419-6769-5
eBook Packages: EngineeringEngineering (R0)