Don’t Compare Averages

  • Holger Bast
  • Ingmar Weber
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3503)


We point out that for two sets of measurements, it can happen that the average of one set is larger than the average of the other set on one scale, but becomes smaller after a non-linear monotone transformation of the individual measurements. We show that the inclusion of error bars is no safeguard against this phenomenon. We give a theorem, however, that limits the amount of “reversal” that can occur; as a by-product we get two non-standard one-sided tail estimates for arbitrary random variables which may be of independent interest. Our findings suggest that in the not infrequent situation where more than one cost measure makes sense, there is no alternative other than to explicitly compare averages for each of them, much unlike what is common practice.


Cost Measure Monotone Transformation Hierarchical Dirichlet Process Neural Information Processings System Statistical Natural Language Processing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Manning, C.D., Schütze, H.: Foundations of statistical natural language processing. MIT Press, Cambridge (1999)zbMATHGoogle Scholar
  2. 2.
    Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Sharing clusters among related groups: Hierarchical dirichlet processes. In: Proceedings of the Advances in Neural Information Processings Systems Conference (NIPS 2004), MIT Press, Cambridge (2004)Google Scholar
  3. 3.
    Lavrenko, V., Croft, W.B.: Relevance based language models. In: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR 2001), pp. 120–127. ACM Press, New York (2001)CrossRefGoogle Scholar
  4. 4.
    Mori, S., Nagao, M.: A stochastic language model using dependency and its improvement by word clustering. In: Proceedings of the 17th international conference on Computational linguistics (COLING 1998), pp. 898–904. Association for Computational Linguistics (1998)Google Scholar
  5. 5.
    Grimmett, G., Stirzaker, D.: Probability and Random Processes. Oxford University Press, Oxford (1992)Google Scholar
  6. 6.
    Siegel, A.: Median bounds and their application. Journal of Algorithms 38, 184–236 (2001)zbMATHCrossRefGoogle Scholar
  7. 7.
    Basu, S., Dasgupta, A.: The mean, median and mode of unimodal distributions: A characterization. Theory of Probability and its Applications 41, 210–223 (1997)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Munro, J.I., Paterson, M.S.: Selection and sorting with limited storage. Theoretical Computer Science 12, 315–323 (1980)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Manku, G.S., Rajagopalan, S., Lindsay, B.G.: Approximate medians and other quantiles in one pass and with limited memory. In: Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD 1998), pp. 426–435 (1998)Google Scholar
  10. 10.
    Motulsky, H.: The link between error bars and statistical significance,

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Holger Bast
    • 1
  • Ingmar Weber
    • 1
  1. 1.Max-Planck-Institut für InformatikSaarbrückenGermany

Personalised recommendations