Data Organization

  • Shai Vaingast


A preliminary step to designing and programming an algorithm is gathering data and sorting it. When you first test a thesis or write code to analyze network traffic, only part of the information is readily available; some of the data is still unknown. You make initial estimations based on the first set of data files. As data is gathered, you discover new insights and understandings, resulting in possible changes to the processing script and data gathering application, such as adding a previously unlogged parameter and graphing it over time. Some changes may include data gathering over substantially longer time periods than originally anticipated. Consequently, to accommodate for manageable data files, you reduce the sampling rate, implemented by logging only every nth value. Another plausible scenario is that of parsing log files, where the generating application (e.g., a web server) recently went through a software upgrade that altered the file format and the file name scheme.


Data File File Format File Size Text Editor Binary File 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Copyright information

© Shai Vaingast 2014

Authors and Affiliations

  • Shai Vaingast
    • 1
  1. 1.Ganey TikvaIsrael

Personalised recommendations