What Cannot Be Measured Cannot Be Controlled: Gauging Success with A/B Tests
- 1.7k Downloads
The robust measurement of the efficiency of recommendation algorithms is an extremely important factor in the development of recommendation engines. We provide some useful methodical remarks on this topic in this chapter, even though it is not directly connected to the problem of adaptive learning. We further propose a straightforward algorithm to calculate confidence intervals for REs. At the end, we discuss Simpson’s paradox which illustrates the importance of constant environment conditions for testing.