The Network Data Handling War: MySQL vs. NfDump

  • Rick Hofstede
  • Anna Sperotto
  • Tiago Fioreze
  • Aiko Pras
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6164)


Network monitoring plays a crucial role in any network management environment. Especially nowadays, with network speed and load constantly increasing, more and more data needs to be collected and efficiently processed. In highly interactive network monitoring systems, a quick response time from information sources turns out to be a crucial requirement. However, for data sets in the order of several GBs, this goal becomes difficult to achieve. In this paper, we present our operational experience in dealing with large amounts of network data. In particular, we focus on MySQL and NfDump, testing their capabilities under different usage scenarios and increasing data set sizes.


  1. 1.
    Steinder, M., Sethi, A.S.: A survey of fault localization techniques in computer networks. Science of Computer Programming 53(2), 165–194 (2004)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Casey, E.: Network traffic as a source of evidence: tool strengths, weaknesses, and future needs. Digital Investigation 1(1), 28–43 (2004)CrossRefGoogle Scholar
  3. 3.
    Tcpdump/libpcap (April 2010),
  4. 4.
    Claise, B.: Cisco Systems NetFlow Services Export Version 9. RFC 3954, Informational (2004)Google Scholar
  5. 5.
    MySQL (April 2010),
  6. 6.
    PostgreSQL (April 2010),
  7. 7.
    Tcptrace (April 2010),
  8. 8.
    NfDump (April 2010),
  9. 9.
    Liu, X., Heo, J., Sha, L.: Modeling 3-Tiered Web Applications. In: Proc. of the 13th IEEE Int. Symp. on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, pp. 307–310 (2005)Google Scholar
  10. 10.
    Hofstede, R., Fioreze, T.: SURFmap: A Network Monitoring Tool Based on the Google Maps API. In: Application session proc. of the 11th IFIP/IEEE Int. Symp. on Integrated Network Management, pp. 676–690. IEEE Computer Society Press, Los Alamitos (2009)CrossRefGoogle Scholar
  11. 11.
    Li, Y., Slagell, A., Luo, K., Yurcik, W.: CANINE: A combined conversion and anonymization tool for processing NetFlows for security. In: Proc. of 10th Int. Conf. on Telecommunication Systems, Modeling and Analysis (2005)Google Scholar
  12. 12.
    Minarik, P., Dymacek, T.: NetFlow Data Visualization Based on Graphs. In: Goodall, J.R., Conti, G., Ma, K.-L. (eds.) VizSec 2008. LNCS, vol. 5210, pp. 144–151. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    University of Twente (April 2010),
  14. 14.
    Siekkinen, M., Biersack, E.W., Urvoy-Keller, G., Goebel, V., Plagemann, T.: InTraBase: Integrated traffic analysis based on a database management system. In: Proc. of the End-to-End Monitoring Techniques and Services, Washington, DC, USA, pp. 32–46. IEEE Computer Society, Los Alamitos (2005)CrossRefGoogle Scholar
  15. 15.
    Kobayashi, A., Matsubara, D., Kimura, S., Saitou, M., Hirokawa, Y., Sakamoto, H., Ishibashi, K., Yamamoto, K.: A Proposal of Large-Scale Traffic Monitoring System Using Flow Concentrators. In: Kim, Y.-T., Takano, M. (eds.) APNOMS 2006. LNCS, vol. 4238, pp. 53–62. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  16. 16.
    Lim, K.S., Stadler, R.: Real-time views of network traffic using decentralized management. In: Proc. of the 9th IFIP/IEEE Int. Symp. on Integrated Network Management, Nice, France, pp. 119–132 (2005)Google Scholar
  17. 17.
    Quittek, J., Zseby, T., Claise, B., Zander, S.: Requirements for IP Flow Information Export (IPFIX). RFC 3917, Informational (2004)Google Scholar
  18. 18.
    Schwartz, B., Zaitsev, P., Tkachenko, V., Zawodny, J., Lentz, A., Balling, D.J.: High performance MySQL, 2nd edn. O’Reilly, Sebastopol (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Rick Hofstede
    • 1
  • Anna Sperotto
    • 1
  • Tiago Fioreze
    • 1
  • Aiko Pras
    • 1
  1. 1.Faculty of Electrical Engineering, Mathematics and Computer Science, Design and Analysis of Communications Systems (DACS)University of Twente, Centre for Telematics and Information TechnologyEnschedeThe Netherlands

Personalised recommendations