Skip to main content
SpringerLink
Log in
Book cover

Econometric Evaluation of Socio-Economic Programs pp 1–47Cite as

An Introduction to the Econometrics of Program Evaluation

  • Chapter

Part of the book series: Advanced Studies in Theoretical and Applied Econometrics ((ASTA,volume 49))

Abstract

It is common practice for policymakers to perform ex post evaluation of the impact of economic and social programs via evidence-based statistical analysis. This effort is mainly devoted to measure the “causal effects” of an intervention on the part of an external authority (generally, a local or national government) on a set of subjects (people, companies, etc.) targeted by the program. Evidence-based evaluation is progressively becoming an integral part of many policies worldwide. The main motivation resides in the fact that, when a public authority chooses to support private entities by costly interventions, a responsibility towards taxpayers is assumed. This commitment, constitutionally recognized in several countries, draws upon the principle that, since many alternative uses of the same amount of money are generally possible, any misuse of it is seen as waste, especially under severe budget constraints.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    A wide range of literature witnesses this relevance. See reviews and books such as: Heckman (2000); Heckman et al. (2000); Blundell and Costa Dias (2002); Shadish et al. (2002); Cobb-Clark and Crossley (2003); Imbens (2004); Lee (2005); Morgan and Winship (2007); Imbens and Wooldridge (2009); Angrist and Pischke (2008); Millimet et al. (2008); Imbens and Wooldridge (2009); Cerulli (2010); Guo and Fraser (2010); Wooldridge (2002, Chap. 18); Wooldridge (2010, Chap. 21).

  2. 2.

    Probably more explicit in this direction might be the recent developments in the field of “continuous treatment” where the treatment variable x assumes a continuous form. In this case, although the setting is very close to the traditional econometric regression, the counterfactual approach provides new insights on the meaning of causal parameters, as in the definition and estimation of the Average Partial Effect (Wooldridge 2001) or of the Average Potential Outcome (Hirano and Imbens 2004).

  3. 3.

    For an in–depth study of this subject, see: Imbens and Wooldridge (2009, pp. 17–18); Frölich and Melly (2013); Abadie et al. (2002); Chernozhukov and Hansen (2005). See also Frölich and Melly (2010) for a Stata implementation.

  4. 4.

    See Lee (2005, pp. 12–13) for a simple numerical example of such a situation.

  5. 5.

    Key references are: Manski (1993, 2013), Rosenbaum (2007), Sobel (2006), Hudgens and Halloran (2008), Tchetgen-Tchetgen and VanderWeele (2010), Cerulli (2014a).

  6. 6.

    Observe that the lower bound of ATE(x) is equal to the lower bound of p(Y 1 = 1 | x) minus the upper bound of p(Y 0 = 1 | x), while the upper bound of ATE(x) is equal to the upper bound of p(Y 1 = 1 | x) minus the lower bound of p(Y 0 = 1 | x).

  7. 7.

    Observe that an estimation of p(D = 0 | x) is obtained as \( \left[1-\widehat{p}\left(D=1\Big|\mathbf{x}\right)\right] \).

References

  • Abadie, A., Angrist, J., & Imbens, G. (2002). Instrumental variables estimates of the effect of subsidized training on the quantiles of trainee earnings. Econometrica, 70, 91–117.

    Article  Google Scholar 

  • Althaus, C., Bridgman, P., & Davis, G. (2007). The Australian policy handbook. Sydney: Allen & Unwin.

    Google Scholar 

  • Angrist, J. D. (1991). Instrumental variables estimation of average treatment effects in econometrics and epidemiology (NBER Technical Working Papers No. 0115).

    Google Scholar 

  • Angrist, J. D., & Imbens, G. W. (1995). Two-stage least squares estimation of average causal effects in models with variable treatment intensity. Journal of the American Statistical Association, 90, 431–442.

    Article  Google Scholar 

  • Angrist, J. D., & Pischke, J. S. (2008). Mostly harmless econometrics: An empiricist’s companion. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Blundell, R., & Costa Dias, M. (2002). Alternative approaches to evaluation in empirical microeconomics. Portuguese Economic Journal, 1, 91–115.

    Article  Google Scholar 

  • Cameron, A. C., & Trivedi, P. K. (2005). Microeconometrics: Methods and applications. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Cattaneo, M. D. (2010). Efficient semiparametric estimation of multi-valued treatment effects under ignorability. Journal of Econometrics, 155, 138–154.

    Article  Google Scholar 

  • Cerulli, G. (2010). Modelling and measuring the effect of public subsidies on business R&D: Critical review of the economic literature. Economic Record, 86, 421–449.

    Article  Google Scholar 

  • Cerulli, G. (2014a). Identification and estimation of treatment effects in the presence of neighbourhood interactions (Working Paper Cnr–Ceris, N° 04/2014).

    Google Scholar 

  • Cerulli, G. (2014b). CTREATREG: Stata module for estimating dose–response models under exogenous and endogenous treatment (Working Paper Cnr–Ceris, N° 05/2014).

    Google Scholar 

  • Chernozhukov, V., & Hansen, C. (2005). An IV model of quantile treatment effects. Econometrica, 73, 245–261.

    Article  Google Scholar 

  • Cobb-Clark, D. A., & Crossley, T. (2003). Econometrics for evaluations: An introduction to recent developments. Economic Record, 79, 491–511.

    Article  Google Scholar 

  • Cooley, T. F., & LeRoy, S. F. (1985). Atheoretical macroeconometrics: A critique. Journal of Monetary Economics, 16, 283–308.

    Article  Google Scholar 

  • Frölich, M. (2004). Programme evaluation with multiple treatments. Journal of Economic Surveys, 18, 181–224.

    Article  Google Scholar 

  • Frölich, M., & Melly, B. (2010). Estimation of quantile treatment effects with Stata. Stata Journal, 10(3), 423–457.

    Google Scholar 

  • Frölich, M., & Melly, B. (2013). Unconditional quantile treatment effects under endogeneity. Journal of Business & Economic Statistics, 31(3), 346–357.

    Article  Google Scholar 

  • Guo, S., & Fraser, M.-W. (2010). Propensity score analysis. Statistical methods and applications. Thousand Oaks, CA: SAGE.

    Google Scholar 

  • Heckman, J. J. (2000). Causal parameters and policy analysis in economics: A twentieth century retrospective. Quarterly Journal of Economics, 115, 45–97.

    Article  Google Scholar 

  • Heckman, J. J. (2001). Micro data, heterogeneity, and the evaluation of public policy: Nobel lecture. Journal of Political Economy, 109, 673–748.

    Article  Google Scholar 

  • Heckman, J., Ichimura, H., Smith, J., & Todd, P. (1998). Characterizing selection bias using experimental data. Econometrica, 66, 1017–1098.

    Article  Google Scholar 

  • Heckman, J. J., Lalonde, R., & Smith, J. (2000). The economics and econometrics of active labor markets programs. In A. Ashenfelter & D. Card (Eds.), Handbook of labor economics (Vol. 3). New York: Elsevier.

    Google Scholar 

  • Hirano, K., & Imbens, G. (2004). The propensity score with continuous treatments. In A. Gelman & X. L. Meng (Eds.), Applied Bayesian modeling and causal inference from incomplete-data perspectives (pp. 73–84). New York: Wiley.

    Google Scholar 

  • Holland, P. (1986). Statistics and causal inference (with discussion). Journal of the American Statistical Association, 81, 945–970.

    Article  Google Scholar 

  • Hoover, K. D. (2001). Causality in macroeconomics. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Hudgens, M. G., & Halloran, M. E. (2008). Toward causal inference with interference. Journal of the American Statistical Association, 103(482), 832–842.

    Article  Google Scholar 

  • Husted, J. A., Cook, R. J., Farewell, V. T., & Gladman, D. D. (2000). Methods for assessing responsiveness: A critical review and recommendations. Journal of Clinical Epidemiology, 53, 459–468.

    Article  Google Scholar 

  • Imai, K., & Van Dyk, D. (2004). Causal inference with general treatment regimes: Generalizing the propensity score. Journal of the American Statistical Association, 99, 854–866.

    Article  Google Scholar 

  • Imbens, G. W. (2000). The role of the propensity score in estimating dose-response functions. Biometrika, 87, 706–710.

    Article  Google Scholar 

  • Imbens, G. W. (2004). Nonparametric estimation of average treatment effects under exogeneity: A review. The Review of Economics and Statistics, 86(1), 4–29.

    Article  Google Scholar 

  • Imbens, G. W., & Wooldridge, J. M. (2009). Recent developments in the econometrics of program evaluation. Journal of Economic Literature, 47, 5–86.

    Article  Google Scholar 

  • Koopmans, T. C. (1947). Measurement without theory. The Review of Economic Statistics, 29(3), 161–172.

    Article  Google Scholar 

  • Koopmans, T. C. (1949). Reply to Rutledge Vining. The Review of Economic Statistics, 31, 86–91.

    Article  Google Scholar 

  • Lee, M. J. (2005). Micro-econometrics for policy, program and treatment effects. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Lucas, R. E. (1976). Econometric policy evaluation: A critique. In K. Brunner & A. H. Meltzer (Eds.), The Phillips curve and labor markets (pp. 19–46). Amsterdam: North-Holland.

    Google Scholar 

  • Lucas, R. E. (1980). Methods and problems in business cycle theory. Journal of Money, Credit and Banking, 12, 696–715.

    Article  Google Scholar 

  • Lucas, R. E., & Sargent, T. J. (1981). After Keynesian macroeconomics. In R. E. Lucas & T. J. Sargent (Eds.), Rational expectations and econometric practice (pp. 295–319). Minneapolis, MN: University of Minnesota Press.

    Google Scholar 

  • Manski, C. F. (1993). Identification of endogenous social effects: The reflection problem. The Review of Economic Studies, 60(3), 531–542.

    Article  Google Scholar 

  • Manski, C. F. (2003). Partial identification of probability distributions. New York: Springer.

    Google Scholar 

  • Manski, C. F. (2013). Identification of treatment response with social interactions. The Econometrics Journal, 16(1), S1–S23.

    Article  Google Scholar 

  • Manski, C. F., Sandefur, G. D., McLanahan, S., & Powers, D. (1992). Alternative estimates of the effect of family structure during adolescence on high school graduation. Journal of the American Statistical Association, 87, 417.

    Article  Google Scholar 

  • Millimet, D., Smith, J., & Vytlacil, E. (Eds.). (2008). Advances in econometrics, Vol 21. Modelling and evaluating treatment effects in econometrics. Amsterdam: JAI Press, Elsevier.

    Google Scholar 

  • Moran, M., Rein, M., & Goodin, R. E. (Eds.). (2008). The Oxford handbook of public policy. Oxford: Oxford University Press.

    Google Scholar 

  • Morgan, S. L., & Winship, C. (2007). Counterfactuals and causal inference: Methods and principles for social research. New York: Cambridge University Press.

    Book  Google Scholar 

  • Neyman, J. (1923). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Translated in Statistical Science, 5(1990), 465–480.

    Google Scholar 

  • Pearl, J. (2000). Causality: Models, reasoning and inference. Cambridge: Cambridge University Press.

    Google Scholar 

  • Pearl, J. (2009). Causal inference in statistics: An overview. Statistics Surveys, 3, 96–146.

    Article  Google Scholar 

  • Potì, B., & Cerulli, G. (2011). Evaluation of firm R&D and innovation support: New indicators and the ex-ante prediction of ex-post additionality. Research Evaluation, 20(1), 19–29.

    Article  Google Scholar 

  • Rosenbaum, P. R. (2007). Interference between units in randomized experiments. Journal of the American Statistical Association, 102(477), 191–200.

    Article  Google Scholar 

  • Rosenbaum, P., & Rubin, D. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55.

    Article  Google Scholar 

  • Rothman, K. J., Greenland, S., & Lash, T. L. (2008). Modern epidemiology. Philadelphia: Lippincott Williams & Wilkins.

    Google Scholar 

  • Rubin, D. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66, 688–701.

    Article  Google Scholar 

  • Rubin, D. B. (1977). Assignment to treatment group on the basis of a covariate. Journal of Educational Statistics, 2, 1–26.

    Google Scholar 

  • Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Annals of Statistics, 6(1), 34–58.

    Article  Google Scholar 

  • Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.

    Google Scholar 

  • Sims, C. A. (1980). Macroeconomics and reality. Econometrica, 48, 1–48.

    Article  Google Scholar 

  • Sims, C. A. (1996). Macroeconomics and methodology. Journal of Economic Perspectives, 10, 105–120.

    Article  Google Scholar 

  • Sobel, M. E. (2006). What do randomized studies of housing mobility demonstrate? Causal inference in the face of interference. Journal of the American Statistical Association, 101(476), 1398–1407.

    Article  Google Scholar 

  • Tchetgen-Tchetgen, E. J., & VanderWeele, T. J. (2010). On causal inference in the presence of interference. Statistical Methods in Medical Research, 21(1), 55–75.

    Article  Google Scholar 

  • Trochim, W., & Donnelly, J. P. (2007). The research methods knowledge base. Mason, OH: Thomson.

    Google Scholar 

  • Vining, R. (1949a). Koopmans on the choice of variables to be studied and the methods of measurement. The Review of Economics and Statistics, 31, 77–86.

    Article  Google Scholar 

  • Vining, R. (1949b). Rejoiner. The Review of Economics and Statistics, 31, 91–96.

    Article  Google Scholar 

  • Wooldridge, J. M. (2001). Unobserved heterogeneity and estimation of average partial effects. Michigan State University, Department of Economics, mimeo.

    Google Scholar 

  • Wooldridge, J. M. (2002). Econometric analysis of cross section and panel data. Cambridge: MIT Press.

    Google Scholar 

  • Wooldridge, J. M. (2010). Econometric analysis of cross section and panel data. Cambridge, MA: MIT Press. Chapter 21.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research Institute on Sustainable Economic Growth, CNR-IRCrES National Research Council of Italy, Roma, Italy

    Giovanni Cerulli

Authors
  1. Giovanni Cerulli
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Cerulli, G. (2015). An Introduction to the Econometrics of Program Evaluation. In: Econometric Evaluation of Socio-Economic Programs. Advanced Studies in Theoretical and Applied Econometrics, vol 49. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-46405-2_1

Download citation

Publish with us

Policies and ethics

Search

Navigation