Knowing is Half the Battle, or Is It? A Randomized Experiment of the Impact of Supplemental Notification Letters on Placement Exam Participation, Preparation, and Performance
Community college students often are unaware of the stakes involved in their performance on placement exams, used to sort the students into their first math and English courses. In our randomized experiment of approximately 13,000 newly admitted community college students, we test whether informing students, via a supplemental notification letter, about the implications of placement exam scores influences their placement exam participation, preparation, and performance. We also test whether message framing (loss-framed vs. gain-framed) alters the effect of the information on study outcomes. We find that neither type of message has an effect on placement exam preparation or performance. We find limited evidence to suggest that the loss-framed message may have a small effect on the avoidance of placement exams by increasing the likelihood of students’ submitting alternative evidence of academic preparation. Overall, our evidence suggests that students’ prior knowledge about the ramifications of placement exams is not enough to improve performance. Colleges should consider alternative means of assessing college readiness, such as measures of high school achievement, and should provide students who still have to take placement exams with detailed information about test content and study materials before testing.
KeywordsCommunity college Assessment Placement Developmental education Remedial Intervention Experiment Randomized trial
An earlier version of this study was presented at the 2016 annual meeting of the Association for the Study of Higher Education in Columbus, Ohio. The authors thank the Editor and anonymous referees of the journal Research in Higher Education for feedback on this work. In addition, the authors thank the administration of the study college for support of this investigation, and also thank Susan Dynarski (University of Michigan) for the suggestion that ultimately led to this investigation.
- Bahr, P. R. (2016). Replacing placement tests in Michigan’s community colleges. Ann Arbor: University of Michigan.Google Scholar
- Bahr, P. R., Fagioli, L. P., Hetts, J., Hayward, C., Willett, T., Lamoree, D., Newell, M. A., Sorey, K., & Baker, R. B. (in press). Improving placement accuracy in California’s community colleges using multiple measures of high school achievement. Community College Review.Google Scholar
- Bahr, P. R., & Gross, J. (2016). Community colleges. In M. N. Bastedo, P. G. Altbach, & P. J. Gumport (Eds.), American higher education in the 21st century (4th ed., pp. 462–502). Baltimore: John Hopkins University.Google Scholar
- Belfield, C. R., & Crosta, P. M. (2012). Predicting success in college: The importance of placement tests and high school transcripts (CCRC Working Paper No. 42). New York: Columbia University.Google Scholar
- Burdman, P. (2012). Where to begin? The evolving role of placement exams for students starting college. Boston: Jobs for the Future. Retrieved from Jobs for the Future website: http://www.jff.org/publications/where-begin-evolving-role-placement-exams-students-starting-college.
- Fay, M. P., Bickerstaff, S., & Hodara, M. (2013). Why students do not prepare for math placement exams: Student perspectives (CCRC Brief No. 57). New York: Columbia University.Google Scholar
- Fields, R., & Parsad, B. (2012). Tests and cut scores used for student placement in postsecondary education: Fall 2011. Washington, DC: National Assessment Governing Board. Retrieved from https://www.nagb.org/content/nagb/assets/documents/commission/researchandresources/test-and-cut-scores-used-for-student-placement-in-postsecondary-education-fall-2011.pdf.
- Ginder, S. A., Kelly-Reid, J. E., & Mann, F. B. (2015). Enrollment and Employees in Postsecondary Institutions, Fall 2014; and Financial Statistics and Academic Libraries, Fiscal Year 2014: First Look (Provisional Data) (NCES 2016-005). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved from http://nces.ed.gov/pubsearch.
- Hodara, M., Jaggars, S. S., & Karp, M. M. (2012). Improving developmental education assessment and placement: Lessons from community colleges across the country (CCRC Working Paper No. 51). New York: Columbia University.Google Scholar
- Lenhart, A. (2012). Teens, smartphones & texting. Pew Internet & American Life Project, 21, 1–34.Google Scholar
- March, J. G. (1994). Primer on decision making: How decisions happen. New York: Simon and Schuster.Google Scholar
- Safran, S., & Visher, M. G. (2010). Case studies of three community colleges: The policy and practice of assessing and placing students in developmental education courses. New York: Columbia University.Google Scholar
- Scott-Clayton, J. (2012). Do high-stakes exams predict college success? (CCRC Working Paper No. 41). New York: Columbia University.Google Scholar
- Shadish, W. R., Cook, T. D., & Campbell, T. D. (2002). Experimental and quasi-experimental design for generalized causal inference. Boston: Houghton Mifflin.Google Scholar
- Venezia, A., Bracco, K. R., & Nodine, T. (2010). One-shot deal? Students’ perceptions of assessment and course placement in California’s community colleges. San Francisco, CA: WestEd.Google Scholar
- Yeaton, W. H., & Moss, B. G. (in press). A multiple-design, experimental strategy: Academic probation warning letter’s impact on student achievement. The Journal of Experimental Education.Google Scholar