# How much evidence should one collect?

- 313 Downloads
- 1 Citations

## Abstract

A number of philosophers of science and statisticians have attempted to justify conclusions drawn from a finite sequence of evidence by appealing to results about what happens if the length of that sequence tends to infinity. If their justifications are to be successful, they need to rely on the finite sequence being either indefinitely increasing or of a large size. These assumptions are often not met in practice. This paper analyzes a simple model of collecting evidence and finds that the practice of collecting only very small sets of evidence before taking a question to be settled is rationally justified. This shows that the appeal to long run results can be used neither to explain the success of actual scientific practice nor to give a rational reconstruction of that practice.

## Keywords

Philosophy of science Evidence Rational choice Formal epistemology Bayesian epistemology Sequential decision problems## Notes

### Acknowledgments

Thanks to Kevin Zollman, Kevin Kelly, Liam Bright, Adam Brodie, and an anonymous referee for valuable comments and discussion.

## References

- Boring, E. G. (1950).
*A history of experimental psychology*(2nd ed.). New Jersey: Prentice-Hall.Google Scholar - Casella, G., & Berger, R. L. (2001).
*Statistical inference*(2nd ed.). Belmont: Duxbury.Google Scholar - DeGroot, M. H. (2004).
*Optimal statistical decisions*. New Jersey: Wiley.CrossRefGoogle Scholar - Earman, J. (1992).
*Bayes or bust? A critical examination of Bayesian confirmation theory*. Cambridge: MIT Press.Google Scholar - Edwards, W., Lindman, H., & Savage, L. J. (1963). Bayesian statistical inference for psychological research.
*Psychological Review*,*70*(3), 193–242.CrossRefGoogle Scholar - Friedman, M. (1979). Truth and confirmation.
*The Journal of Philosophy*,*76*(7), 361–382.CrossRefGoogle Scholar - Hempel, C. G. (1945a). Studies in the logic of confirmation (I.).
*Mind*,*54*(213), 1–26.CrossRefGoogle Scholar - Hempel, C. G. (1945b). Studies in the logic of confirmation (II.).
*Mind*,*54*(214), 97–121.CrossRefGoogle Scholar - Howson, C., & Urbach, P. (1989).
*Scientific reasoning: the Bayesian approach*. La Salle: Open Court.Google Scholar - Kelly, K. T. (1991). Reichenbach, induction, and discovery.
*Erkenntnis*,*35*(1/3), 123–149.Google Scholar - Kelly, K. T. (1996).
*The logic of reliable inquiry*. Oxford: Oxford University Press.Google Scholar - Lai, T. L. (1981). Asymptotic optimality of invariant sequential probability ratio tests.
*The Annals of Statistics*,*9*(2), 318–333.CrossRefGoogle Scholar - Liu, Y., & Blostein, S. D. (1992). Optimality of the sequential probability ratio test for nonstationary observations.
*IEEE Transactions on Information Theory*,*38*(1), 177–182.CrossRefGoogle Scholar - Olesko, K. M., & Holmes, F. L. (1993). Experiment, quantification, and discovery: Helmholtz’s early physiological researches, 1843–50. In D. Cahan (Ed.),
*Hermann von Helmholtz and the foundations of Nineteenth-century science*, Chap. 2 (pp. 50–108). Berkeley: University of California Press.Google Scholar - Peirce, C. S. (1931 [1878]). How to make our ideas clear. In C. Hartstone & P. Weiss (Eds.),
*Collected Papers of Charles Sanders Peirce*(Vol. V, pp. 5.388–5.410). Cambridge: Harvard University Press.Google Scholar - Popper, K. (1959).
*The logic of scientific discovery*. London: Hutchinson.Google Scholar - Reichenbach, H. (1938).
*Experience and prediction: An analysis of the foundations and the structure of knowledge*. Chicago: The University of Chicago Press.CrossRefGoogle Scholar - Schurz, G. (2008). The meta-inductivist’s winning strategy in the prediction game: A new approach to Hume’s problem.
*Philosophy of Science*,*75*(3), 278–305.CrossRefGoogle Scholar - Stein, C. (1946). A note on cumulative sums.
*The Annals of Mathematical Statistics*,*17*(4), 498–499.CrossRefGoogle Scholar - von Helmholtz, H. L. F. (1850). Messungen über den zeitlichen Verlauf der Zuckung animalischer Muskeln und die Fortpflanzungsgeschwindigkeit der Reizung in den Nerven
*Archiv für Anatomie, Physiologie und wissenschaftliche Medicin*(pp. 276–364).Google Scholar - Wald, A. (1947).
*Sequential analysis*. New York: Wiley.Google Scholar - Wald, A., & Wolfowitz, J. (1948). Optimum character of the sequential probability ratio test.
*The Annals of Mathematical Statistics*,*19*(3), 326–339.CrossRefGoogle Scholar