Abstract
Though randomized control trials continue to serve as the “gold standard” of causal inference, they are neither feasible nor desirable in numerous instances. Even in the absence of randomized trials, higher education researchers have at their disposal several statistical tools for estimating causal relationships. One such method is difference-in-differences, a powerful and intuitive approach to causal evaluation that exploits variation in the timing and coverage of policies. The method lends itself well to studying higher education policies and initiatives, as these frequently diffuse over time and across space in ways that may permit for causal inference. Difference-in-differences has become one of the most widely used methods for causal inference in higher education research. We use this chapter to introduce new researchers to this method with an overview of difference-in-differences models, common threats to their validity, and robustness checks. We then present extensions of the method, including event study models and variation in treatment timing. We illustrate these methods throughout the chapter by analyzing the effect of hurricanes on enrollment at affected colleges using data from the Integrated Postsecondary Education Data System and provide Stata code for replication of the analysis.
Keywords
- Policy evaluation
- Causal inference
- Quasi-experimental design
- Natural experiments
- Counterfactuals
- Difference-in-differences
- Event study models
- Parallel trends assumptions
- Fixed effects models
- Variation in treatment timing
- Heterogeneous effects
- Robustness checks
- Multiple comparison groups
- Clustered and bootstrapped standard errors
- Hurricane Katrina
- Geography of college choice
- Enrollment trends
This is a preview of subscription content, log in via an institution.
References
Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105(490), 493–505.
Andrews, R. J., DesJardins, S., & Ranchhod, V. (2010). The effects of the Kalamazoo promise on college choice. Economics of Education Review, 29(5), 722–737.
Angrist, J. (2004). American education research changes tack. Oxford Review of Economic Policy, 20(2), 198–212.
Angrist, J. D., & Pischke, J. S. (2009). Mostly harmless econometrics: An empiricist's companion. Princeton: Princeton University Press.
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455.
Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W., & Wager, S. (2019). Synthetic difference in differences (No. w25532). National Bureau of Economic Research.
Ashenfelter, O. (1978). Estimating the effect of training programs on earnings. The Review of Economics and Statistics, 60(1), 47–57.
Athey, S., & Imbens, G. W. (2018). Design-based analysis in difference-in-differences settings with staggered adoption (No. w24963). National Bureau of Economic Research.
Baker, D. J. (2019). Pathways to racial equity in higher education: Modeling the antecedents of state affirmative action bans. American Educational Research Journal. Advance online publication. https://doi.org/10.3102/0002831219833918.
Belasco, A. S., Rosinger, K. O., & Hearn, J. C. (2015). The test-optional movement at America’s selective liberal arts colleges: A boon for equity or something else? Educational Evaluation and Policy Analysis, 37(2), 206–223.
Bertrand, M., Duflo, E., & Mullainathan, S. (2004). How much should we trust differences-in- differences estimates? The Quarterly Journal of Economics, 119(1), 249–275.
Betts, J. R., & McFarland, L. L. (1995). Safe port in a storm: The impact of labor market conditions on community college enrollments. Journal of Human Resources, 30(4), 741–765.
Bielby, R. M., House, E., Flaster, A., & DesJardins, S. L. (2013). Instrumental variables: Conceptual issues and an application considering high school course taking. In M. B. Paulsen (Ed.), Higher education: Handbook of theory and research (Vol. XXVIII, pp. 263–321). Dordrecht: Springer.
Brewer, M., Crossley, T. F., & Joyce, R. (2018). Inference with difference-in-differences revisited. Journal of Econometric Methods, 7(1), 1–16.
Bumphus, W. G., & Royal, A. M. (2007). In our own words: Disaster recovery post-Katrina. In A. W. Villadsen & G. E. De los Santos (Eds.), Hoping for the best while preparing for the worst: Disasters, emergencies and the community college (pp. 24–30). Phoenix: League for Innovation in the Community College.
Charles, K., Hurst, E., & Notowidigdo, M. (2018). Housing booms and busts, labor market opportunities, and college attendance. American Economic Review, 108(10), 2947–2994.
Conley, T. G., & Taber, C. R. (2011). Inference with “`difference in differences” with a small number of policy changes. The Review of Economics and Statistics, 93(1), 113–125.
Cortes, K. E. (2010). Do bans on affirmative action hurt minority students? Evidence from the Texas Top 10% Plan. Economics of Education Review, 29(6), 1110–1124.
Dache-Gerbino, A. (2018). College desert and oasis: A critical geographic analysis of local college access. Journal of Diversity in Higher Education, 11(2), 97–116.
Daw, J. R., & Hatfield, L. A. (2018). Matching and regression to the mean in difference-in-differences analysis. Health Services Research, 53(6), 4138–4156.
Deryugina, T., Kawano, L., & Levitt, S. (2018). The economic impact of Hurricane Katrina on its victims: Evidence from individual tax returns. American Economic Journal: Applied Economics, 10(2), 202–233.
DeWaard, J., Curtis, K., & Fussell, E. (2016). Population recovery in New Orleans after Hurricane Katrina: Exploring the potential role of stage migration in migration systems. Population and Environment, 37(4), 449–463.
Dynarski, S. M. (2003). Does aid matter? Measuring the effect of student aid on college attendance and completion. American Economic Review, 93(1), 279–288.
Foote, A., & Grosz, M. (2019). The effect of local labor market downturns on postsecondary enrollment and program choice. Education Finance and Policy. Advance online publication.
Freyaldenhoven, S., Hansen, C., & Shapiro, J. M. (2018). Pre-event trends in the panel event-study design (No. w24565). National Bureau of Economic Research.
Furquim, F., & Glasener, K. M. (2017). A quest for equity? Measuring the effect of QuestBridge on economic diversity at selective institutions. Research in Higher Education, 58(6), 646–671.
Furuta, J. (2017). Rationalization and student/school personhood in US college admissions: The rise of test-optional policies, 1987 to 2015. Sociology of Education, 90(3), 236–254.
Fussell, E. (2015). The long-term recovery of New Orleans’ population after Hurricane Katrina. American Behavioral Scientist, 59(10), 1231–1245.
Galea, S., Brewin, C., Gruber, M., Jones, R., King, D., King, L., McNally, R., Ursano, R., Petukhova, M., & Kessler, R. (2007). Exposure to hurricane-related stressors and mental illness after Hurricane Katrina. Archives of General Psychiatry, 64(12), 1427–1434.
Garces, L. M. (2013). Understanding the impact of affirmative action bans in different graduate fields of study. American Educational Research Journal, 50(2), 251–284.
Garces, L. M., & Mickey-Pabello, D. (2015). Racial diversity in the medical profession: The impact of affirmative action bans on underrepresented student of color matriculation in medical schools. The Journal of Higher Education, 86(2), 264–294.
Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. (2016). Impact evaluation in practice. Washington, DC: The World Bank.
Goodman-Bacon, A. (2018). Difference-in-differences with variation in treatment timing (No. w25018). National Bureau of Economic Research.
Groen, J., & Polivka, A. (2008). The effect of Hurricane Katrina on the labor market outcomes of evacuees. American Economic Review, 98(2), 43–48.
Heckman, J. J., & Smith, J. A. (1999). The pre-programme earnings dip and the determinants of participation in a social programme: Implications for simple programme evaluation strategies. The Economic Journal, 109(457), 313–348.
Heller, D. E. (1999). The effects of tuition and state financial aid on public college enrollment. The Review of Higher Education, 23(1), 65–89.
Hemelt, S., & Marcotte, D. (2011). The impact of tuition increases on enrollment at public colleges and universities. Educational Evaluation and Policy Analysis, 33(4), 435–457.
Hillman, N. W. (2017). Geospatial analysis in higher education research. In M. B. Paulsen (Ed.), Higher education: Handbook of theory and research (Vol. XXXII, pp. 529–575).
Hillman, N. W., & Orians, E. L. (2013). Community colleges and labor market conditions: How does enrollment demand change relative to local unemployment rates? Research in Higher Education, 54(7), 765–780.
Hillman, N. W., Tandberg, D. A., & Gross, J. P. K. (2014). Performance funding in higher education: Do financial incentives impact college completions? The Journal of Higher Education, 85(6), 826–857.
Hillman, N. W., Tandberg, D. A., & Fryar, A. H. (2015). Evaluating the impacts of “new” performance funding in higher education. Educational Evaluation and Policy Analysis, 37(4), 501–519.
Hillman, N. W., Fryar, A. H., & Crespín-Trujillo, V. (2018). Evaluating the impact of performance funding in Ohio and Tennessee. American Educational Research Journal, 55(1), 144–170.
Holland, P. W. (1986). Statistics and causal inference. Journal of the American statistical Association, 81(396), 945–960.
Hubbard, D. (2018). The impact of local labor market shocks on college choice: Evidence from plant closings in Michigan. Unpublished manuscript. Retrieved from https://www.aeaweb.org/conference/2019/preliminary/paper/Q67dtN77
Iacus, S. M., King, G., & Porro, G. (2012). Causal inference without balance checking: Coarsened exact matching. Political Analysis, 20(1), 1–24.
Jaquette, O., Kramer, D. A., & Curs, B. R. (2018). Growing the pie? The effect of responsibility center management on tuition revenue. The Journal of Higher Education, 89(5), 637–676.
Jarvie, J., & Landsberg, M. (2018, September 17). When a Category None hurricane is worse than a Category 5. Los Angeles Times. Retrieved from https://www.latimes.com
Johnson, G., & Rainey, S. (2007). Hurricane Katrina’s impact on three historically black colleges and universities (HBCUs): Voices from displaced students. Race, Gender & Class, 14(1), 100–119.
Kahn-Lang, A., & Lang, K. (2019). The promise and pitfalls of differences-in-differences: Reflections on 16 and pregnant and other applications. Journal of Business & Economic Statistics, 1–14.
Kates, R. W., Colten, C. E., Laska, S., & Leatherman, S. P. (2006). Reconstruction of New Orleans after Hurricane Katrina: a research perspective. Proceedings of the National Academy of Sciences, 103(40), 14653–14660.
Kothari, S. P., & Warner, J. B. (2007). Econometrics of event studies. In B. E. Eckbo (Ed.), Handbook of empirical corporate finance (pp. 3–36). North Holland: Elsevier.
Ladd, A., Gill, D., & Marszalek, J. (2007). Riders from the storm: Narratives of relocated New Orleans college students in the aftermath of Hurricane Katrina. Journal of Public Management & Social Policy, 13(2), 51–80.
Lafortune, J., Rothstein, J., & Schanzenbach, D. W. (2018). School finance reform and the distribution of student achievement. American Economic Journal: Applied Economics, 10(2), 1–26.
Lall, R. (2016). How multiple imputation makes a difference. Political Analysis, 24(4), 414–433.
Lapid, P. (2018). Expanding college access: The impact of new universities on local enrollment. Unpublished manuscript. Retrieved from https://www.ocf.berkeley.edu/~palapid/wp-content/uploads/2019/03/newU-SEA-18_11_05.pdf
Lechner, M. (2010). The estimation of causal effects by difference-in-difference methods. Foundations and Trends in Econometrics, 4(3), 165–224.
Li, A. Y., & Kennedy, A. I. (2018). Performance funding policy effects on community college outcomes: Are short-term certificates on the rise? Community College Review, 46(1), 3–39.
Liang, K. Y., & Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73(1), 13–22.
Long, B. T. (2011). The new financial aid policies: Their impact on access and equity for low-income students. In L. M. Stulberg & S. L. Weinberg (Eds.), Diversity in American higher education: Toward a more comprehensive approach. New York: Routledge Books.
Lowe, S., & Rhodes, J. (2012). Community college re-enrollment after Hurricane Katrina. Journal of College Student Retention, 14(2), 227–249.
Mangan, K., & O’Leary, B. (2010). New Orleans colleges 5 years after Katrina. The Chronicle of Higher Education. https://www.chronicle.com/article/New-Orleans-Colleges-5-Years/124107. Accessed 29 May 2019.
McCall, B. P., & Bielby, R. M. (2012). Regression discontinuity design: Recent developments and a guide to practice for researchers in higher education. In J. C. Smart & M. B. Paulsen (Eds.), Higher education: Handbook of theory and research (Vol. XXVII, pp. 249–290). Dordrecht: Springer.
Murnane, R., & Willett, J. (2011). Methods matter: Improving causal inference in educational and social science research. Oxford, UK: Oxford University Press.
Ngo, F., & Astudillo, S. (2019). California DREAM: The impact of financial aid for undocumented community college students. Educational Researcher, 48(1), 5–18.
Reynolds, C. L., & DesJardins, S. L. (2009). The use of matching methods in higher education research: Answering whether attendance at a 2-year institution results in differences in educational attainment. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. XXIV, pp. 47–97). Dordrecht: Springer.
Rios-Aguilar, C., & Titus, M. (2018). Spatial research in higher education: Expanding understanding and identifying inequities. New Directions in Institutional Research, 2018(180), 5–10.
Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688.
Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469), 322–331.
Sacerdote, B. (2012). When the saints go marching out: Long-term outcomes for student evacuees from Hurricanes Katrina and Rita. American Economic Journal: Applied Economics, 4(1), 109–135.
Savoie, E. J., O’Neil, R. M., & Rabban, D. M. (2007). Report of an AAUP special committee: Hurricane Katrina and New Orleans universities. Academe, 93(3), 59–126.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random and nonrandom assignments. Journal of the American Statistical Association, 103(484), 1334–1344.
St. Clair, T., & Cook, T. D. (2015). Difference-in-differences methods in public finance. National Tax Journal, 68(2), 319–338.
Stange, K. (2015). Differential pricing in undergraduate education: Effects on degree production by field. Journal of Policy Analysis and Management, 34(1), 107–135.
Strauss, V. (2018, September 4). The real story of New Orleans and its charter schools. The Washington Post. Retrieved from https://www.washingtonpost.com
Strezhnev, A. (2018). Semiparametric weighting estimators for multi-period difference-in-differences designs. Working paper. Retried from https://www.antonstrezhnev.com/research
Tandberg, D. A., & Hillman, N. W. (2014). State higher education performance funding: Data, outcomes, and policy implications. Journal of Education Finance, 29(3), 222–243.
Turley, R. (2009). College proximity: Mapping access to opportunity. Sociology of Education, 82(2), 126–146.
U.S. Census Bureau. (2018a). Metropolitan and micropolitan: About. Retrieved from https://www.census.gov/programs-surveys/metro-micro/about.html
U.S. Census Bureau. (2018b). Metropolitan and micropolitan: Guidance for data users. Retrieved from https://www.census.gov/programs-surveys/metro-micro/guidance.html
U.S. Census Bureau. (2019). Resident population in Orleans Parish, LA [LAORLE0POP]. Retrieved from FRED, Federal Reserve Bank of St. Louis; https://fred.stlouisfed.org/series/LAORLE0POP. 13 Apr 2019.
U.S. Department of Education. (2019). Integrated postsecondary education data system. https://nces.ed.gov/ipeds/datacenter/Data.aspx
U.S. Department of Homeland Security. (2019). OpenFEMA dataset: Disaster declarations summaries – V1. Retrieved from https://www.fema.gov/openfema-dataset-disaster-declarations-summaries-v1
Vigdor, J. (2008). The economic aftermath of Hurricane Katrina. Journal of Economic Perspectives, 22(4), 135–154.
Wolfers, J. (2006). Did unilateral divorce laws raise divorce rates? A reconciliation and new results. American Economic Review, 96(5), 1802–1820.
Zhang, L. (2010). The use of panel data models in higher education policy studies. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. XXV, pp. 307–349). Dordrecht: Springer.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Appendix Code
Appendix Code
The following Stata code produced many of the tables and charts included in this chapter. Because of space constraints, we have omitted various parts of the code such as data management steps and much of the analysis beyond Hurricane Katrina; we are happy to share complete .do files by request.
// open data for analysis use C:\Desktop\did_handbook_data.dta// install the following user-written programs: ssc install lgraph, replace ssc install blindschemes, replace ssc install estout, replace ssc install coefplot, replace// figure 1: fall enrollment for colleges in new orleans lgraph total_enroll year if treat==1, scheme(plottig) ylabel(0(500)3500) /// legend(off) loptions(1 lcolor(black) lpat(line) mcolor(black)) ytitle("Mean fall enrollment") xtitle("")// figure 2: fall enrollment for colleges in new orleans and other southern states lgraph total_enroll year treat, scheme(plottig) ylabel(0(500)3500) /// legend(on order(1 "Southern cities" 2 "New Orleans") position(6)) /// loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) ///lpat(line) mcolor(black)) ytitle("Mean fall enrollment") xtitle("")// table 3: did means table table treat post if (comparison==1 | treat==1), c(mean total_enroll) f(%10.0fc)// table 4: canonical did regression with different standard errors and covariates ∗ top panel: canonical did regression with different standard errors reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1) estimates store table4_a // no se adjustment reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1), robust estimates store table4_b // robust s.e. reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1), /// cluster(unitid) estimates store table4_c // cluster s.e. ∗ bottom panel: canonical did regression (cluster s.e.) with covariates and fe global controls "tuition1 metro_ue_rate" reg total_enroll i.treat i.post i.treat#i.post $controls if (comparison==1|treat==1), /// cluster(unitid) estimates store table4_d // controls only areg total_enroll i.treat i.post i.treat#i.post i.year if _est_table4_d==1 & /// (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table4_e // fixed effects only areg total_enroll i.treat i.post i.treat#i.post $controls i.year if _est_table4_d==1 &/// (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table4_f // controls and fe estout table4_a table4_b table4_c table4_d table4_e table4_f, cells(b(star fmt(2) /// label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) /// stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2))// table 5: canonical did with multiple comparison groups, without and with controls ∗ create regional neighbor gen neighbor = 1 if inlist(stabbr,"AR","MS","TX") & sreb==1 ∗ create ps matched logit treat total_enroll if sreb==1 & year<2005 predict double ps ssc install psmatch2, replace psmatch2 treat, outcome(total_enroll) pscore(ps) egen ps_match = min(_weight), by(unitid) ∗ top panel (without controls) reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1), /// cluster(unitid) estimates store table5_a // new orleans vs sreb reg total_enroll i.treat i.post i.treat#i.post, cluster(unitid) estimates store table5_b // new orleans vs nationwide reg total_enroll i.treat i.post i.treat#i.post if (neighbor==1 | treat==1), /// cluster(unitid) estimates store table5_c // new orleans vs neighbor reg total_enroll i.treat i.post i.treat#i.post if (ps_match==1 | treat==1), /// cluster(unitid) estimates store table5_d // new orleans vs matched ∗ bottom panel (with controls) reg total_enroll i.treat i.post i.treat#i.post $controls if (comparison==1|treat==1),/// cluster(unitid) estimates store table5_e // new orleans vs sreb reg total_enroll i.treat i.post i.treat#i.post $controls, cluster(unitid) estimates store table5_f // new orleans vs nationwide reg total_enroll i.treat i.post i.treat#i.post $controls if (neighbor==1|treat==1),///cluster(unitid) estimates store table5_g // new orleans vs neighbor reg total_enroll i.treat i.post i.treat#i.post $controls if (ps_match==1|treat==1),/// cluster(unitid) estimates store table5_h // new orleans vs matched estout table5_a table5_b table5_c table5_d table5_e table5_f table5_g table5_h, ///cells(b(star fmt(2) label(Coef.)) se(par fmt(2) label(std.errors))) starlevels///( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2))// figure 6: enrollment trend for multiple comparison groups lgraph total_enroll year treat if (sreb==1 | treat==1), scheme(plottig) /// ylabel(0(500)3500) name(sreb, replace) legend(on order(1 "SREB" 2 "New Orleans") ///position(6)) loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) lpat(line)///mcolor(black)) ytitle("Mean fall enrollment") xtitle("") lgraph total_enroll year treat, scheme(plottig) ylabel(0(500)3500) name(us, replace) ///legend(on order(1 "U.S." 2 "New Orleans") position(6)) loptions(0 lcolor(black) ///lpat(dash) m(square); 1 lcolor(black) lpat(line) mcolor(black)) ytitle("Mean fall ///enrollment") xtitle("") lgraph total_enroll year treat if sreb==1 & (neighbor==1 | treat==1), scheme(plottig) ///ylabel(0(500)3500) name(neigh, replace) legend(on order(1 "Neighbors" 2 "New Orleans")///position(6)) loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) lpat(line)///mcolor(black)) ytitle("Mean fall enrollment") xtitle("") lgraph total_enroll year treat if sreb==1 & (ps_match==1 | treat==1), scheme(plottig)///ylabel(0(500)4500) name(psm, replace) legend(on order(1 "PS Matched" 2 "New Orleans")///position(6)) loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) lpat(line)///mcolor(black)) ytitle("Mean fall enrollment") xtitle("") graph combine sreb us neigh psm, name(combined, replace)// table 6: did regression with state-specific trends encode stabbr, gen(stn) areg total_enroll i.treat i.post i.treat#i.post i.stn##c.year if (comparison==1 | treat==1), absorb(stn) cluster(unitid) estimates store table6_a // state x year trends areg total_enroll i.treat i.post i.treat#i.post i.year if (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table6_b // fe only areg total_enroll i.treat i.post i.treat#i.post i.stn##c.year i.year if (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table6_c //state x years trends and fe estout table6_a table6_b table6_c, cells(b(star fmt(2) label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2))//table 7: placebo test for change to treatment timing ∗generate placebo years gen placebo_2003 = 1 if year>=2003 gen placebo_2004 = 1 if year>=2004 gen placebo_2005 = 1 if year>=2005 gen placebo_2006 = 1 if year>=2006 gen placebo_2007 = 1 if year>=2007 recode placebo_2003-placebo_2007 (.=0) ∗ top panel (without controls) reg total_enroll i.treat i.placebo_2003 i.treat#i.placebo_2003 if year<2005, /// cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect estimates store p_2003_noco reg total_enroll i.treat i.placebo_2004 i.treat#i.placebo_2004 if year<2005, /// cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect est sto p_2004_noco reg total_enroll i.treat i.placebo_2005 i.treat#i.placebo_2005, cluster(unitid) est sto p_2005_noco reg total_enroll i.treat i.placebo_2006 i.treat#i.placebo_2006 if year>2005, /// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2006_noco reg total_enroll i.treat i.placebo_2007 i.treat#i.placebo_2007 if year>2005, /// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2007_noco ∗ bottom panel (with control) reg total_enroll i.treat i.placebo_2003 i.treat#i.placebo_2003 $controls if year<2005,///cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect est sto p_2003_co reg total_enroll i.treat i.placebo_2004 i.treat#i.placebo_2004 $controls if year<2005,/// cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect est sto p_2004_co reg total_enroll i.treat i.placebo_2005 i.treat#i.placebo_2005 $controls, cluster(unitid) est sto p_2005_co reg total_enroll i.treat i.placebo_2006 i.treat#i.placebo_2006 $controls if year>2005,/// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2006_co reg total_enroll i.treat i.placebo_2007 i.treat#i.placebo_2007 $controls if year>2005,/// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2007_co estout p_2003_noco p_2004_noco p_2005_noco p_2006_noco p_2007_noco,cells(b(star fmt(2)///label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010)/// stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2)) estout p_2003_co p_2004_co p_2005_co p_2006_co p_2007_co, cells(b(star fmt(2) ///label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010)///stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2))// table 8: placebo test for non-equivalent outcome ∗ top panel (in-state tuition) reg tuition1 i.treat i.post i.treat#i.post if (comparison==1 | treat==1), cluster(unitid)estimates store table8_a // new orleans vs sreb reg tuition1 i.treat i.post i.treat#i.post, cluster(unitid) estimates store table8_b // new orleans vs nationwide reg tuition1 i.treat i.post i.treat#i.post if (neighbor==1 | treat==1), cluster(unitid) estimates store table8_c // new orleans vs neighbor reg tuition1 i.treat i.post i.treat#i.post if (ps_match==1 | treat==1), cluster(unitid) estimates store table8_d // new orleans vs matched ∗ bottom panel (out-of-state tuition) reg tuition3 i.treat i.post i.treat#i.post if (comparison==1 | treat==1), cluster(unitid) estimates store table8_e // new orleans vs sreb reg tuition3 i.treat i.post i.treat#i.post, cluster(unitid) estimates store table8_f // new orleans vs nationwide reg tuition3 i.treat i.post i.treat#i.post if (neighbor==1 | treat==1), cluster(unitid) estimates store table8_g // new orleans vs neighbor reg tuition3 i.treat i.post i.treat#i.post if (ps_match==1 | treat==1), cluster(unitid) estimates store table8_h // new orleans vs matched estout table8_a table8_b table8_c table8_d table8_e table8_f table8_g table8_h,/// cells(b(star fmt(2) label(Coef.)) se(par fmt(2) label(std.errors))) starlevels///( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2))// table 9: did regression for multiple hurricanes areg total_enroll i.treat_mh i.year if inc==1, vce(cluster unitid) absorb(unitid)// table 10: event study results (hurricane katrina) ∗ create adoption year of treatment and limit to five-year pre/post period gen adopt_delta=2005-year if treat==1 gen within_5=(adopt_delta==. | inrange(adopt_delta, -5, 5)) ∗create event study lag (here’s how to do it in a loop) forvalues i=1(1)5 { gen predelta_`i'=(adopt_delta==`i') label var predelta_`i' "-`i'" } ∗ create event study lead (here’s how to do it one by one) gen postdelta_0=(adopt_delta==0) label var postdelta_0 "0" gen postdelta_1=(adopt_delta==-1) label var postdelta_1 "1" gen postdelta_2=(adopt_delta==-2) label var postdelta_2 "2" gen postdelta_3=(adopt_delta==-3) label var postdelta_3 "3" gen postdelta_4=(adopt_delta<=-4) label var postdelta_4 "4" gen postdelta_5=(adopt_delta<=-5) label var postdelta_5 "5" areg total_enroll predelta_5 predelta_4 predelta_3 predelta_2 postdelt∗ i.year if ///within_5==1 & year<=2010, vce(cluster unitid) absorb(unitid) estimates store fig7// figure 7: event study estimates (hurricane katrina) coefplot fig7, keep ( predelta_5 predelta_4 predelta_3 predelta_2 postdelta_0 ///postdelta_1 postdelta_2 postdelta_3 postdelta_4 postdelta_5) vertical xlabel ///(, angle(vertical)) xtitle("Years since hurricane (0)")ytitle("Estimated effect")///yline(0, lcolor(black)) scheme(plottig)// table 11: event study results (all hurricanes) ∗ create new within_5 & pre/post indicators because they should be relative to each hurricane cap drop adopt_delta within_5 predelta∗ postdelta∗ gen adopt_delta=year-first_hurr if hurr_ever==1 ∗create event study lag (here’s how to do it in a loop) forvalues i=1(1)5 { gen predelta_`i'=(adopt_delta==`i') label var predelta_`i' "-`i'" } ∗ create event study lead (here’s how to do it one by one) gen postdelta_0=(adopt_delta==0) label var postdelta_0 "0" gen postdelta_1=(adopt_delta==-1) label var postdelta_1 "1" gen postdelta_2=(adopt_delta==-2) label var postdelta_2 "2" gen postdelta_3=(adopt_delta==-3) label var postdelta_3 "3" gen postdelta_4=(adopt_delta<=-4) label var postdelta_4 "4" gen postdelta_5=(adopt_delta<=-5) label var postdelta_5 "5" ∗limit to obs within 5 yrs of a hurricane. gen within_5=(adopt_delta==. | inrange(adopt_delta, -5, 5)) ∗ coefficient only areg total_enroll predelta_2 predelta_3 predelta_4 predelta_5 postdelt∗ i.year if ///within_5==1 & inc==1, vce(cluster unitid) absorb(unitid) ∗ adding state-specific trends areg total_enroll predelta_2 predelta_3 predelta_4 predelta_5 postdelt∗ i.year ///i.year##c.stn if within_5==1 & inc==1, vce(cluster unitid) absorb(unitid)// figure 8: event study estimates (all hurricanes) coefplot fig8a, keep ( predelta_5 predelta_4 predelta_3 predelta_2 postdelta_0 /// postdelta_1 postdelta_2 postdelta_3 postdelta_4 postdelta_5) vertical xlabel ///(, angle(vertical)) xtitle("Years since hurricane (0)") ytitle("Estimated effect")///yline(0, lcolor(black)) scheme(plotplain)
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this entry
Cite this entry
Furquim, F., Corral, D., Hillman, N. (2020). A Primer for Interpreting and Designing Difference-in-Differences Studies in Higher Education Research. In: Perna, L. (eds) Higher Education: Handbook of Theory and Research. Higher Education: Handbook of Theory and Research, vol 35. Springer, Cham. https://doi.org/10.1007/978-3-030-11743-6_5-1
Download citation
DOI: https://doi.org/10.1007/978-3-030-11743-6_5-1
Received:
Accepted:
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-11743-6
Online ISBN: 978-3-030-11743-6
eBook Packages: Springer Reference EducationReference Module Humanities and Social SciencesReference Module Education