Advertisement

Research in Higher Education

, Volume 49, Issue 8, pp 758–775 | Cite as

Using Propensity Scores for Estimating Causal Effects: A Study in the Development of Moral Reasoning

  • Heidi E. Grunwald
  • Matthew J. Mayhew
Article

Abstract

The purpose of this study was to illustrate the use of propensity scores for creating comparison groups, partially controlling for pretreatment course selection bias, and estimating the treatment effects of selected courses on the development of moral reasoning in undergraduate students. Specifically, we used a sample of convenience for comparing differences in moral reasoning development scores among students enrolled in intergroup dialogue, service learning, psychology and philosophy courses with those of an introductory sociology course. Adopting a propensity score approach included reviewing the empirical literature for its guidance in substantiating the reasons for including pretreatment variables (i.e., pretreatment course-taking behaviors, race, sex, political identification, need for cognition, major, age, pretreatment moral reasoning scores) in our analysis, measuring these variables, and reducing them into a single composite propensity score for each student in our analytic sample. This score then served as the basis for creating a new comparison group and for allowing us to estimate unbiased (or less biased) course-related treatment effects on moral reasoning development. Implications for higher education researchers are discussed.

Keywords

Propensity scores Causal analysis Selection bias Moral reasoning development 

Notes

Acknowledgements

The authors gratefully acknowledge the Wabash Center for Inquiry into the Liberal Arts for their generous support of this research.

References

  1. Achen, C. H. (1986). The statistical analysis of quasi-experiments. Berkeley, CA: University of California Press.Google Scholar
  2. Attewell, P. A., Lavin, D. E., Domina, T., & Levey, T. (2006). New evidence on college remediation. Journal of Higher Education, 77(5), 886–924.CrossRefGoogle Scholar
  3. Blechman, E., Maurice, A., Buecker, B., & Helberg, C. (2000). Can mentoring or skill training reduce recidivism? Observational study with propensity analysis. Prevention Science, 1(3), 139–155.CrossRefGoogle Scholar
  4. Cacioppo, J. T., Petty, R. E., Feinstein, J. A., & Jarvis, W. B. G. (1996). Dispositional differences in cognitive motivation: The life and times of individuals varying in need for cognition. Psychological Bulletin, 119(2), 197–253.CrossRefGoogle Scholar
  5. Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of the need for cognition scale. Journal of Personality Assessment, 48, 306–307.CrossRefGoogle Scholar
  6. Caldwell, M., Skeem, J., Salkin, R., & Van Rybroek, G. (2006). Treatment response of adolescent offenders with psychopathy-like features. Criminal Justice and Behavior, 33(5), 571–596.CrossRefGoogle Scholar
  7. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton-Mifflin.Google Scholar
  8. Dehejia, R. H., & Wahba, S. (1999). Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs. Journal of the American Statistical Association, 94(448), 1053–1062.CrossRefGoogle Scholar
  9. Feldman, K. A., & Newcomb, T. M. (1969). The impact of college on students. San Francisco: Jossey-Bass.Google Scholar
  10. Hansen, B. B. (2004). Full matching in an observational study of coaching for the SAT. Journal of the American Statistical Association, 99, 609–618.CrossRefGoogle Scholar
  11. Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161.CrossRefGoogle Scholar
  12. Heckman, J. J., Ichimura, H., Smith, J., & Todd, P. (1998a). Characterizing selection bias using experimental data. Econometrica, 66, 1017–1098.CrossRefGoogle Scholar
  13. Heckman, J. J., Ichimura, H., & Todd, P. (1998b). Matching as an econometric evaluation estimator. Review of Economic Studies, 65, 261–294.CrossRefGoogle Scholar
  14. Hodges, K., & Grunwald, H. (2005). The use of propensity scores to evaluate outcomes for community clinics: Identification of an exceptional home-based program. Journal of Behavioral Health Services Research, 32(3), 294–305.CrossRefGoogle Scholar
  15. Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–968.CrossRefGoogle Scholar
  16. Hume, D. (1740). A treatise on human nature. New York: Oxford University Press.Google Scholar
  17. Jones, A. S., D’Agostino, R. B., Gondolf, E., & Heckert, A. (2004). Assessing the effect of batterer program completion on reassault using propensity scores. Journal of Interpersonal Violence, 19(9), 1002–1020.CrossRefGoogle Scholar
  18. King, P. M., & Mayhew, M. J. (2004). Theory and research on the development of moral reasoning among college students. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. XIX, pp. 375–440). The Netherlands: Kluwer Academic Publishers.Google Scholar
  19. LaLonde, J. R. (1986). Evaluating the econometric evaluation of training programs with experimental data. American Economic Review, 76(4), 604–620.Google Scholar
  20. Lewis, D. (1973). Counterfactuals. Oxford: Blackwell.Google Scholar
  21. Mayhew, M. J. (2005). How curricular content and educational practices influence the development of moral reasoning in traditionally-aged undergraduate students. Unpublished doctoral dissertation, University of Michigan.Google Scholar
  22. Mill, J.S. (1843). A system of logic. 2 volumes. London: Parker.Google Scholar
  23. Pascarella, E. T., Wolniak, G. C., & Pierson, C. T. (2003). Explaining student growth in college when you don’t think you are. Journal of College Student Development, 44, 122–126.CrossRefGoogle Scholar
  24. Raudenbush, S. W. (2004). What are value-added models estimating and what does this imply for statistical practice? Journal of Educational and Behavioral Statistics, 29, 121–129.CrossRefGoogle Scholar
  25. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage Publications.Google Scholar
  26. Raudenbush, S. W., & Willms, J. D. (1995). The estimation of school effects. Journal of Educational and Behavioral Statistics, 20, 307–335.Google Scholar
  27. Rest, J. R. (1979). Development in judging moral issues. Minneapolis, MN: University of Minnesota.Google Scholar
  28. Rest, J. R., Narvaez, D., Bebeau, M., & Thoma, S. J. (1999). Postconventional moral thinking: A neo-Kohlbergian approach. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  29. Rosenbaum, P.R. (1984). From association to causation in observational studies: The role of tests of strongly ignorable treatment assignment. Journal of American Statistical Association, (79), 41–48.CrossRefGoogle Scholar
  30. Rosenbaum, P. R. (1986). Dropping out of high school in the United States: An observational study. Journal of Educational Statistics, 11, 207–224.CrossRefGoogle Scholar
  31. Rosenbaum, P. R. (1988). Sensitivity analysis for matching and multiple controls. Biometrika, 75, 577–581.CrossRefGoogle Scholar
  32. Rosenbaum, P. R., & Rubin, D.B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55.CrossRefGoogle Scholar
  33. Rosenbaum, P.R., & Rubin, D.B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516–524.CrossRefGoogle Scholar
  34. Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701.CrossRefGoogle Scholar
  35. Rubin, D. B. (1980). Bias reduction using Mahalanobis-metric matching. Biometrics, 36, 293–298.CrossRefGoogle Scholar
  36. Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine, 127, 757–763.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Temple UniversityPhiladelphiaUSA
  2. 2.New York UniversityNew YorkUSA

Personalised recommendations