Application-Oriented Evaluation of Measurement Estimation

Abstract

The design and operation of telecommunication networks often requires measurement-based decisions. Examples of such decisions are QoS-based routing, where the measurement of the utilized link capacity influences the path selection, or cache location that uses measurements of average TCP throughput. However, the variability of measurements of network conditions makes it difficult to use them for such decisions. In QoS routing, decisions need to be stable in order to avoid packet reordering; in cache location, a small change in the measurements can lead to a completely different location decision. Therefore, estimation algorithms are used to smooth the measurements. The quality of decisions based on estimations depends on how close the estimation is to real conditions, and on how variable it is. There is a trade-off between these two objectives that makes it difficult to choose an appropriate estimation method. In this paper, an approach that uses multi-criteria analysis to evaluate the quality of an estimation algorithm is introduced. It is shown how different applications require different criteria and how the preferences of these criteria can be set by the algorithm designer. Some example estimation algorithms are evaluated on synthetic and real traffic traces using the proposed approach.