Skip to main content
SpringerLink
Log in

Planning an Observational Study

  • Chapter
Observational Studies

Part of the book series: Springer Series in Statistics ((SSS))

  • 3739 Accesses

Abstract

Many observational studies do not succeed in providing tangible, enduring, and convincing evidence about the effects caused by treatments, and those that do succeed often exhibit great care in their design. Particularly at the early stages of design, this care consists of choices that determine the circumstances of the study and the data to be collected. A convincing observational study is the result of active observation, an active search for those rare circumstances in which tangible evidence may be obtained to distinguish treatment effects from the most plausible biases. In an experiment, treatment effects are seen clearly because the environment is tightly controlled, whereas in a compelling observational study, control is, to a large extent, replaced by choice—the environment is carefully chosen.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Allison, P. D. (1990) Change scores as dependent variables in regression analyses. In: Sociological Methodology, C. C. Clogg, ed., Oxford: Basil Blackwell, pp. 93–114.

    Google Scholar 

  • Altonji, J. G. and Dunn, T. A. (1996) Using siblings to estimate the effect of school quality on wages. Review of Economics and Statistics, 77, 665–671.

    Article  Google Scholar 

  • Anderson, E. (2000) Code of the Streets: Decency, Violence and the Moral Life of the Inner City. New York: Norton.

    Google Scholar 

  • Angrist, J. D. and Krueger, A. B. (1999) Empirical strategies in labor economics. In: Handbook of Labor Economics, III, New York: Elsevie, Chapter 23.

    Google Scholar 

  • Angrist, J. D. and Lavy, V. (1999) Using Maimonides’ rule to estimate the effect of class size on scholastic achievement. Quarterly Journal of Economics, 533–575.

    Google Scholar 

  • Ashenfelter, O. A. and Krueger, A. B. (1994) Estimates of the economic return to schooling from a new sample of twins. American Economic Review, 84, 1157–1173.

    Google Scholar 

  • Becker, H. S. (1972) Outsiders: Studies in the Sociology of Deviance. New York: The Free Press.

    Google Scholar 

  • Behrman, J., Rosenzweig, M., and Taubman, P. (1996) College choice and wages: Estimates using data on female twins. The Review of Economics and Statistics, 672–685.

    Google Scholar 

  • Bosk, C. L. (1981) Forgive and Remember: Managing Medical Failure. Chicago: University of Chicago Press.

    Google Scholar 

  • Box, G. E. P. (1966) The use and abuse of regression. Technometrics, 8, 625–629.

    Article  MathSciNet  MATH  Google Scholar 

  • Bronars, S. G. and Grogger, J. (1994) The economic consequences of unwed motherhood: Using twin births as a natural experiment. American Economic Review, 84, 1141–1156.

    Google Scholar 

  • Bross, I. D. J. (1960). Statistical criticism. Cancer, 13, 394–400. Reprinted in: The Quantitative Analysis of Social Problems, E. Tufte, ed., Reading, MA: Addison-Wesley, pp. 97–108.

    Article  Google Scholar 

  • Campbell, D. T. (1957) Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54, 297–312.

    Article  Google Scholar 

  • Campbell, D. T. (1984, 1988) Can we be scientific in applied social science? Evaluation Studies Review Annual, 9, 26–48. Reprinted in: D. Campbell, Methodology and Epistemology for Social Science: Selected Papers. Chicago: University of Chicago Press, pp. 315–333.

    Google Scholar 

  • Campbell, D. and Stanley, R. (1963) Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally.

    Google Scholar 

  • Card, D. (1990) The impact of the Mariel Boatlift on the Miami labor market. Industrial and Labor Relations Review, 43, 245–257.

    Article  Google Scholar 

  • Card, D., Katz, L., and Krueger, A. (1994) Comment. Industrial and Labor Relations Review, 48, 487–496.

    Article  Google Scholar 

  • Card, D. and Krueger, A. (1994) Minimum wages and employment: A case study of the fast-food industry in New Jersey and Pennsylvania. American Economic Review, 84, 772–793.

    Google Scholar 

  • Card, D. and Krueger, A. (1995) Myth and Measurement: The New Economics of the Minimum Wage. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Card, D. and Krueger, A. (2000) Minimum wages and employment: A case study of the fast-food industry in New Jersey and Pennsylvania: Reply. American Economic Review, 90, 1397–1420.

    Article  Google Scholar 

  • Chamberlin, T. C. (1890, 1965) The method of multiple working hypotheses. Originally in Science 1890, 15, 92. Reprinted in Science 1965, 148, 754–759.

    Google Scholar 

  • Cook, T. D., Campbell, D. T. and Peracchio, L. (1990) Quasi-experimentation. In: Handbook of Industrial and Organizational Psychology, M. Dunnette and L. Hough, Palo Alto, eds., CA: Consulting Psychologists Press, Chapter 9, pp. 491–576.

    Google Scholar 

  • Cook, T. D. and Shadish, W. R. (1994) Social experiments: Some developments over the past fifteen years. Annual Review of Psychology, 45, 545–580.

    Article  Google Scholar 

  • Dawes, R. (1996) The purpose of experiments: Ecological validity versus comparing hypotheses. Behavioral and Brain Sciences, 19, 20.

    Article  Google Scholar 

  • Deere, D., Murphy, K., and Welch, F. (1995) Employment and the 1990–1991 minimum-wage hike. American Economic Review, 85, 232–237.

    Google Scholar 

  • Diggle, P. J., Liang, K.Y., and Zeger, S. L. (1994) Analysis of Longitudinal Data. New York: Oxford University Press.

    Google Scholar 

  • Duff, C. (1996) New minimum wage makes few waves: Employers offset 50-cent raise with minor shifts. Wall Street Journal, 20 November 1996, pp. 2–4.

    Google Scholar 

  • Engle, R., Hendry, D., and Richard, J. (1983) Exogeneity. Econometrica, 51, 277–304.

    Article  MathSciNet  MATH  Google Scholar 

  • Estroff, S. E. (1985) Making It Crazy: An Ethnography of Psychiatric Clients in an American Community. Berkeley, CA: University of California Press.

    Book  Google Scholar 

  • Feyerabend, P. (1968) How to be a good empiricist-a plea for tolerance in matters epistemological. In: The Philosophy of Science (Oxford Readings in Philosophy), P. H. Nidditch, ed., New York: Oxford University Press, pp. 12–39.

    Google Scholar 

  • Feyerabend, P. (1975) Against Method. London: Verso.

    Google Scholar 

  • Fisher, R. A. (1935) The Design of Experiments. Edinburgh: Oliver and Boyd.

    Google Scholar 

  • Freedman, D. (1997) From association to causation via regression. Advances in Applied Mathematics, 18, 59–110.

    Article  MathSciNet  MATH  Google Scholar 

  • French, S. A., Story, M., and Jeffery, R. W. (2001) Environmental influences on eating and physical activity. Annual Review of Public Health, 22, 309–335.

    Article  Google Scholar 

  • Friedman, M. (1953) The methodology of positive economics. In: Essays in Positive Economics, Chicago: University of Chicago Press, pp. 3–43.

    Google Scholar 

  • Gastwirth, J. L., Krieger, A. M., and Rosenbaum, P. R. (1997) Hypotheticals and hypotheses. American Statistician, 51, 120–121.

    Google Scholar 

  • Giddens, A. (1979) Central Problems in Social Theory. Berkeley: University of California Press.

    Google Scholar 

  • Holland, P. (1993) Which comes first, cause or effect? In: A Handbook for Data Analysis in the Behavioral Sciences: Methodological Issues, G. Keren and C. Lewis, eds., Hillsdale, NJ: Lawrence Erlbaum, pp. 273–282.

    Google Scholar 

  • Holland, P. & Rubin, D. (1983) On Lord’s paradox. In: Principles of Psychological Measurement: A Festschrift for Frederic Lord, H. Wainer and S. Messick, eds., Hillsdale, NJ: Lawrence Erlbaum, pp. 3–25.

    Google Scholar 

  • Joffe, M., Hoover, D., Jacobson, L., Kingsley, L., Chmiel, J., Visscher, B., and Robins, J. (1998) Estimating the effect of zidovudine on Kaposi’s sarcoma from observational data using a rank preserving structural failure-time model. Statistics in Medicine, 17, 1073–1102.

    Article  Google Scholar 

  • Kessler, R. C. (1997) The effects of stressful life events on depression. Annual Review of Psychology, 48, 191–214.

    Article  Google Scholar 

  • Kuczmarski, R. J., Flegal, K. M., Campbell, S. M., and Johnson, C. L. (1994) Increasing prevalence of overweight among US adults. The National Health and Nutrition Examination Surveys, 1960 to 1991. Journal of the American Medical Association, 272, 205–211

    Article  Google Scholar 

  • Lakatos, I. (1970) Falsification and the methodology of scientific research programs. In: Criticism and the Growth of Knowledge, I. Lakatos and A. Musgrave, eds., New York: Cambridge University Press, pp. 91–196. Reprinted in: I. Lakatos (1978) Philosophical Papers, Volume 1, New York, Cambridge University Press, pp. 8–101.

    Google Scholar 

  • Lakatos, I. (1981) History of science and its rational reconstructions. In: Scientific Revolutions, I. Hacking, ed., New York: Oxford University Press, pp. 107–127. Reprinted from Boston Studies in the Philosophy of Science, VIII, 1970.

    Google Scholar 

  • Lehman, D., Wortman, C., and Williams, A. (1987) Long-term effects of losing a spouse or a child in a motor vehicle crash. Journal of Personality and Social Psychology, 52, 218–231.

    Article  Google Scholar 

  • Li, Y., Propert, K. J., and Rosenbaum, P. R. (2001) Balanced risk set matching. Journal of the American Statistical Association, 96, September, to appear..

    Google Scholar 

  • Lichtenstein, P., Gatz, M., Pedersen, N., Berg, S., and McClearn, G. (1996) A co-twin-control study of response to widowhood. Journal of Gerontology: Psychological Sciences, 51B, 279–289.

    Article  Google Scholar 

  • Manski, C. (1999) Comment. Statistical Science, 14, 279–281.

    Google Scholar 

  • Meehi, P. (1978) Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806–834. Reprinted in: P. Meehl, Selected Philosophical and Methodological Papers, Minneapolis: University of Minnesota Press, 1991, pp. 1–42.

    Article  Google Scholar 

  • Meyer, B. D. (1995) Natural and quasi-experiments in economics. Journal of Business and Economic Statistics, 13, 151–161.

    Google Scholar 

  • Meyer, M. and Fienberg, S, eds. (1992) Assessing Evaluation Studies: The Case of Bilingual Education Strategies. Washington, DC: National Academy Press.

    Google Scholar 

  • Morton, D., Saah, A., Silberg, S., Owens, W., Roberts, M., and Saah, M. (1982) Lead absorption in children of employees in a lead-related industry. American Journal of Epidemiology, 115, 549–555.

    Google Scholar 

  • Neumark, D. and Wascher, W. (1992 Employment effects of minimum and subminimum wages: Panel data on state minimum wage laws. Industrial and Labor Relations Review, 46, 55–81.

    Article  Google Scholar 

  • Neumark, D. and Wascher, W. (1994) Reply. Industrial and Labor Relations Review, 48, 497–512.

    Article  Google Scholar 

  • Peirce, C. S. (1903) On selecting hypotheses. In: Collected Papers of Charles Sanders Peirce, C. Hartshorne and P. Weiss, eds., Volume 5, Cambridge, MA: Harvard University Press, 1960, pp. 413–422.

    Google Scholar 

  • Peto, R., Pike, M., Armitage, P., Breslow, N., Cox, D., Howard, S., Mantel, N., McPherson, K., Peto, J., and Smith, P. (1976) Design and analysis of randomized clinical trials requiring prolonged observation of each patient, I: Introduction and design. British Journal of Cancer, 34, 585–612.

    Article  Google Scholar 

  • Platt, J. (1964) Strong inference. Science, 146, 347–352.

    Article  Google Scholar 

  • Polanyi, M. (1946, 1964) Science, Faith and Society. New York: Oxford University Press. Reprinted, Chicago: University of Chicago Press.

    Google Scholar 

  • Popper, K. (1965) Conjectures and Refutations. New York: Harper and Row.

    Google Scholar 

  • Popper, K. (1968) The Logic of Scientific Discovery. New York: Harper and Row. (English translation of Popper’s 1934 Logik der Forschung.)

    Google Scholar 

  • Putnam, H. (1995) Pragmatism. Oxford: Blackwell.

    Google Scholar 

  • Quine, W. (1951) Two dogmas of empiricism. Philosophical Review. Reprinted in: W. Quine (1980) From a Logical Point of View, Cambridge, MA: Harvard University Press, pp. 20–46.

    Google Scholar 

  • Robins, J. (1989) The control of confounding by intermediate variables. Statistics in Medicine, 8, 679–701.

    Article  Google Scholar 

  • Robins, J. (1992 Estimation of the time-dependent accelerated failure time model in the presence of confounding factors. Biometrika, 79, 321–334.

    Article  MathSciNet  MATH  Google Scholar 

  • Robins, J. (1998) Correction for non-compliance in equivalence trials. Statistics in Medicine, 17, 269–302.

    Article  Google Scholar 

  • Robins, J. (1999) Comment. Statistical Science, 14, 281–293.

    Google Scholar 

  • Robins, J., Rotnitzky, A. and Zhao, L. (1995) Analysis of semiparametric regression models for repeated outcomes in the presence of missing data. Journal of the American Statistical Association, 90,106–121.

    Article  MathSciNet  MATH  Google Scholar 

  • Rosenbaum, P. R. (1984) The consequences of adjustment for a concomitant variable that has been affected by the treatment. Journal of the Royal Statistical Society, Series A, 147, 656–666.

    Article  Google Scholar 

  • Rosenbaum, P. R. (1986) Dropping out of high school in the United States: An observational study. Journal of Educational Statistics, 11, 207 – 224.

    Article  Google Scholar 

  • Rosenbaum, P. R. (1999) Choice as alternative to control in observational studies (with discussion) . Statistical Science, 14, 259–304.

    Article  MATH  Google Scholar 

  • Rosenbaum, P. R. (2001a) Stability in the absence of treatment. Journal of the American Statistical Association, 96, 210–219.

    Article  MathSciNet  MATH  Google Scholar 

  • Rosenbaum, P. R. (2001b) Replicating effects and biases. American Statistician, 55, 223–227.

    Article  MathSciNet  Google Scholar 

  • Rosenzweig, M. and Wolpin, K. (1980) Testing the quantity-quality fertility model: The use of twins as a natural experiment. Econometrica, 48, 227–240.

    Article  MATH  Google Scholar 

  • Rosenzweig, M. R. and Wolpin, K. I. (2000) Natural “natural experiments” in economics. Journal of Economic Literature, 38, 827–874.

    Article  Google Scholar 

  • Rubin, D. B. (1991) Practical implications of modes of statistical inference for causal inference and the critical role of the assignment mechanism. Biometrics, 47, 1213–1234.

    Article  MathSciNet  MATH  Google Scholar 

  • Samuelson, P. (1947, 1983) Foundations of Economic Analysis. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Shadish, W. R. and Cook, T. D. (1999) Comment-Design rules: More steps toward a complete theory of quasi-experimentation. Statistical Science, 14, 294–300.

    Google Scholar 

  • Shadish, W. R., Cook, T. D., and Campbell, D. T. (2002) Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton-Mifflin.

    Google Scholar 

  • Shafer, G. (1996) The Art of Causal Conjecture. Cambridge, MA: MIT Press.

    MATH  Google Scholar 

  • Sobel, M. (1995) Causal inference in the social and behavioral sciences. In: Handbook of Statistical Modelling for the Social and Behavioral Sciences, G. Arminger, C. Clogg, and M. Sobel, eds., New York: Plenum, pp. 1–38.

    Google Scholar 

  • Sowell, T. (1995) Repealing the law of gravity. Forbes, 22 May, p. 82.

    Google Scholar 

  • Susser, M. (1987) Falsification, verification and causal inference in epidemiology: Reconsideration in the light of Sir Karl Popper’s philosophy. In: M. Susser, Epidemiology, Health and Society: Selected Papers, New York: Oxford, pp. 82–93.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, The Wharton School, University of Pennsylvania, 19104-6302, Philadelphia, PA, USA

    Paul R. Rosenbaum

Authors
  1. Paul R. Rosenbaum
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer Science+Business Media New York

About this chapter

Cite this chapter

Rosenbaum, P.R. (2002). Planning an Observational Study. In: Observational Studies. Springer Series in Statistics. Springer, New York, NY. https://doi.org/10.1007/978-1-4757-3692-2_11

Download citation

  • DOI: https://doi.org/10.1007/978-1-4757-3692-2_11

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4419-3191-7

  • Online ISBN: 978-1-4757-3692-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation