How much evidence should one collect?
- 335 Downloads
A number of philosophers of science and statisticians have attempted to justify conclusions drawn from a finite sequence of evidence by appealing to results about what happens if the length of that sequence tends to infinity. If their justifications are to be successful, they need to rely on the finite sequence being either indefinitely increasing or of a large size. These assumptions are often not met in practice. This paper analyzes a simple model of collecting evidence and finds that the practice of collecting only very small sets of evidence before taking a question to be settled is rationally justified. This shows that the appeal to long run results can be used neither to explain the success of actual scientific practice nor to give a rational reconstruction of that practice.
KeywordsPhilosophy of science Evidence Rational choice Formal epistemology Bayesian epistemology Sequential decision problems
Thanks to Kevin Zollman, Kevin Kelly, Liam Bright, Adam Brodie, and an anonymous referee for valuable comments and discussion.
- Boring, E. G. (1950). A history of experimental psychology (2nd ed.). New Jersey: Prentice-Hall.Google Scholar
- Casella, G., & Berger, R. L. (2001). Statistical inference (2nd ed.). Belmont: Duxbury.Google Scholar
- Earman, J. (1992). Bayes or bust? A critical examination of Bayesian confirmation theory. Cambridge: MIT Press.Google Scholar
- Howson, C., & Urbach, P. (1989). Scientific reasoning: the Bayesian approach. La Salle: Open Court.Google Scholar
- Kelly, K. T. (1991). Reichenbach, induction, and discovery. Erkenntnis, 35(1/3), 123–149.Google Scholar
- Kelly, K. T. (1996). The logic of reliable inquiry. Oxford: Oxford University Press.Google Scholar
- Olesko, K. M., & Holmes, F. L. (1993). Experiment, quantification, and discovery: Helmholtz’s early physiological researches, 1843–50. In D. Cahan (Ed.), Hermann von Helmholtz and the foundations of Nineteenth-century science, Chap. 2 (pp. 50–108). Berkeley: University of California Press.Google Scholar
- Peirce, C. S. (1931 ). How to make our ideas clear. In C. Hartstone & P. Weiss (Eds.), Collected Papers of Charles Sanders Peirce (Vol. V, pp. 5.388–5.410). Cambridge: Harvard University Press.Google Scholar
- Popper, K. (1959). The logic of scientific discovery. London: Hutchinson.Google Scholar
- von Helmholtz, H. L. F. (1850). Messungen über den zeitlichen Verlauf der Zuckung animalischer Muskeln und die Fortpflanzungsgeschwindigkeit der Reizung in den Nerven Archiv für Anatomie, Physiologie und wissenschaftliche Medicin (pp. 276–364).Google Scholar
- Wald, A. (1947). Sequential analysis. New York: Wiley.Google Scholar