Realtime Data Mining pp 227-234 | Cite as
What Cannot Be Measured Cannot Be Controlled: Gauging Success with A/B Tests
Chapter
First Online:
- 1.7k Downloads
Abstract
The robust measurement of the efficiency of recommendation algorithms is an extremely important factor in the development of recommendation engines. We provide some useful methodical remarks on this topic in this chapter, even though it is not directly connected to the problem of adaptive learning. We further propose a straightforward algorithm to calculate confidence intervals for REs. At the end, we discuss Simpson’s paradox which illustrates the importance of constant environment conditions for testing.
Keywords
Recommendation Algorithm Recommendation Group Calculate Confidence Interval Recommendation Engine Straightforward Algorithm
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
- [Gea30]Geary, R.C.: The frequency distribution of the quotient of two normal variates. J. R. Stat. Soc. 93(3), 442–446 (1930)CrossRefzbMATHGoogle Scholar
- [Sem81]Semjonow, N. Wissenschaft und Gesellschaft (in Russian). Nauka (1981)Google Scholar
- [SV10]Sieber, H., Volkmer, T.: Ein Konfidenzintervall für den Mehrumsatz bei einem A-B-Test. (in German) Documentation, prudsys AG, 2010Google Scholar
Copyright information
© Springer International Publishing Switzerland 2013