Not adding up: free ridership and spillover calculations in energy efficiency evaluations

Abstract

A key element of evaluation, measurement, and verification (EM&V) studies for energy efficiency programs involves estimation of net energy savings that account for free ridership, spillover, and induced market effects. The existing literature recognizes these effects to be significant and provides detailed guidelines to estimate them. However, there appears to be a disconnect between these guidelines and field evaluations conducted in practice. Our meta-analysis of 120 studies from 2006 to 2018 indicates that most free ridership and spillover estimates are based on survey results and expressed in percentage terms. We note that simply adding these percentages numerically without converting them into a common unit is inaccurate and obscures a program’s true impact. Additionally, there exists wide variations in nomenclature, classification, and methodologies adopted to estimate these metrics across programs and jurisdictions. Our scatterplot analysis of the reviewed EM&V reports indicates that with few exceptions, free ridership and spillover do not necessarily offset each other. We propose an alternative approach to estimate free ridership and spillover in energy units with costs in dollar terms, e.g., as the difference between a program participant’s total willingness-to-pay and the total financial impact of the program’s existence. We also feel that a consistent, transparent, and reliable evaluation methodology to estimate free ridership and spillover effects across programs and jurisdictions based on randomized or quasi-experimental designs will not only improve accuracy but will also have better comparability for informed policy decisions in future.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. Alberini, A., & Bigano, A. (2014). How effective are energy efficiency incentive programs? Evidence from Italian Homeowners. SSRN.

  2. Angelucci, M., & Maro, V. D. (2015). Program evaluation and spillover effects. Retrieved from World Bank Group Policy Research Working Paper: http://documents.worldbank.org/curated/en/137731468171858310/pdf/WPS7243.pdf.

  3. Berg, W., Nowak, S., Kelly, M., Vaidyanathan, S., Shoemaker, M., Chittum, A., et al. (2019). ACEEE Scorecard. Washington, DC: ACEEE.

    Google Scholar 

  4. Boomhower, J., & Davis, L. W. (2014). A credible approach for measuring inframarginal participation in energy efficiency programs. Journal of Public Economics, 113, 67–79.

    Article  Google Scholar 

  5. Felder, F. A., & Athawale, R. (2017). "PACT-a-Mole": the case against using the Program Administrator Test for energy efficiency programs. Energy Efficiency.

  6. Friedrich, K., Eldridge, M., York, D., Witte, P., & Kushler, M. (2009). Saving energy cost-effectively: a national review of the cost of energy saved through utility-sector energy efficiency programs. Washington, DC: American Council for an Energy Efficient Economy.

    Google Scholar 

  7. Grosche, P., & Vance, C. (2009). Willingness to pay for energy conservation and free-ridership on subsidization: evidence from Germany. The Energy Journal, 135–153.

  8. Haeri, H., & Khawaja, M. S. (2012). The trouble with freeriders (p. 40). Public Utilties Fortnightly.

  9. Haugland, T. (1996). Social benefits of financial investment support in energy conservation policy. The Energy Journal, 79–102.

  10. IEA. (2017). International Energy Agency, Energy Efficiency 2017.

  11. Itron, Inc. (2014). Verification of reported program impacts from 2013 empower maryland energy efficiency programs with recommendations to improve future evaluation research. Silver Spring: Itron, Inc..

    Google Scholar 

  12. Jaccard, M. (2010). Paradigms of energy efficiency's cost and their policy implications; deja vu all over again. Modelling the Economics of Greenhouse Gas Mitigation:Summary of a Workshop (pp. 1-19). The National Academic Press.

  13. Jessoe, K., Lade, G. E., Loge, F., & Spang, E. (2017). Spillovers from behavioral interventions: experimental evidence from water and energy use. Berkeley: E2e Project.

    Google Scholar 

  14. Keay, M. (2011). Energy efficiency- should we take it seriously? The Oxford Institute for Energy Studies.

    Google Scholar 

  15. Kraft, J., Rathbun, P., & Le Beau, L. (2010). 2009 Commercial and Industrial Programs Free-ridership and Spillover Study, Final Report. Madison: PA Knowledge Limited.

    Google Scholar 

  16. Li, M., Haeri, H., & Reynolds, A. (2018). The uniform methods project: methods for determining energy efficiency savings for specific measures. Golden: National Renewable Energy Laboratory.

    Google Scholar 

  17. Mahone, D., & Hall, N. (2010). Pursuit of aggressive energy targets: aligning program design and evaluation. Washington: Americam Council for an Energy Efficient Economy.

    Google Scholar 

  18. Massachusetts Executive office of energy and environmental affairs. (2015). Massachusetts Clean Energy and Climate Plan for 2020: 2015 Update. Boston: Massachusetts Executive Office of Energy and Environmental Affairs.

    Google Scholar 

  19. Navigant Consulting, Inc. (2016). Plan Year 6 Summary Evaluation Report. Chicago: Navigant Consulting, Inc..

    Google Scholar 

  20. New York State Joint Utilities. (2018). Department of Public Service New York. Retrieved from Department of Public Service New York: http://www3.dps.ny.gov/W/PSCWeb.nsf/All/72C23DECFF52920A85257F1100671BDD?OpenDocument.

  21. NMR Group, Inc. (2015). Efficiency Maine Retail Lighting Program Overall Evaluation Report. Somerville: NMR Group, Inc..

    Google Scholar 

  22. Opinion Dynamics. (2017). Ameren Illinois Company Electric & Gas Residential, and Commercial and Industrial Energy Efficiency Portfolios – PY8 Summary Evaluation Report. Waltham: Opinion Dynamics.

    Google Scholar 

  23. PWP, Inc. (2017). Current Methods in Free Ridership and Spillover Policy and Estimation Final. North Potomac: PWP, Inc..

    Google Scholar 

  24. Ridge, R., Baker, M., Hall, N., Prahl, R., & Saxonis, W. (2013). Gross is gross and net is net: simple right? Progress in Industrial Ecology- An international Journal, 145-155.

  25. Rosenow, J., & Ray, G. (2012). Evaluating the evaluations:evidence from energy-efficiency programmes in Germany and the UK.

    Google Scholar 

  26. Slote, S., Sherman, M., & Crossley, D. (2014). Energy Efficiency Evaluation, Measurement, and Verification. Regulatory Assistance Project.

    Google Scholar 

  27. State and Local Energy Efficiency Action Network. (2012). Energy Efficiency Program Impact Evaluation Guide. Chicago: Schiller Consulting, Inc..

    Google Scholar 

  28. Vine, E., Prahl, R., Meyers, S., & Turiel, I. (2009). An approach for evaluating the market effects of energy efficiency programs. Energy Efficiency.

  29. Vine, E., Hall, N., Keating, K. M., Kushler, M., & Prahl, R. (2010). Emerging issues in the evaluation of energyefficiency programs: the US experience. Energy Efficiency.

  30. Violette, D. M., & Rathbun, P. (2017). Chapter 21: Estimating Net Savings – Common Practices The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures. Retrieved from NREL Report: https://www.nrel.gov/docs/fy18osti/70472.pdf.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Pranay Kumar.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Froio, Z., Kumar, P. & Felder, F.A. Not adding up: free ridership and spillover calculations in energy efficiency evaluations. Energy Efficiency 13, 991–1005 (2020). https://doi.org/10.1007/s12053-020-09872-6

Download citation

Keywords

  • Energy efficiency
  • Evaluation, measurement, and verification (EM&V)
  • Program design
  • Free ridership
  • Spillover
  • Market effects