Importance of Statistical Evidence in Estimating Valid DEA Scores

  • Darold T. Barnum
  • Matthew Johnson
  • John M. Gleason
Systems-Level Quality Improvement
Part of the following topical collections:
  1. Systems-Level Quality Improvement


Data Envelopment Analysis (DEA) allows healthcare scholars to measure productivity in a holistic manner. It combines a production unit’s multiple outputs and multiple inputs into a single measure of its overall performance relative to other units in the sample being analyzed. It accomplishes this task by aggregating a unit’s weighted outputs and dividing the output sum by the unit’s aggregated weighted inputs, choosing output and input weights that maximize its output/input ratio when the same weights are applied to other units in the sample. Conventional DEA assumes that inputs and outputs are used in different proportions by the units in the sample. So, for the sample as a whole, inputs have been substituted for each other and outputs have been transformed into each other. Variables are assigned different weights based on their marginal rates of substitution and marginal rates of transformation. If in truth inputs have not been substituted nor outputs transformed, then there will be no marginal rates and therefore no valid basis for differential weights. This paper explains how to statistically test for the presence of substitutions among inputs and transformations among outputs. Then, it applies these tests to the input and output data from three healthcare DEA articles, in order to identify the effects on DEA scores when input substitutions and output transformations are absent in the sample data. It finds that DEA scores are badly biased when substitution and transformation are absent and conventional DEA models are used.


Data envelopment analysis DEA Hospital efficiency Hospital quality Input substitution Output transformation 



We gratefully acknowledge financial support for summer research provided by the Dean of the College of Business Administration, University of Illinois at Chicago.


  1. 1.
    Banker, R. D., Estimating the most productive scale size using data envelopment analysis. Eur. J. Oper. Res. 17(1):35–44, 1984.CrossRefGoogle Scholar
  2. 2.
    Charnes, A., Cooper, W. W., Golany, B., Seiford, L., and Stutz, J., Foundations of data envelopment analysis for Pareto-Koopmans efficient empirical production functions. J. Econ. 30(1–2):91–107, 1985. doi: 10.1016/0304-4076(85)90133-2.CrossRefGoogle Scholar
  3. 3.
    Tulkens, H., On FDH efficiency analysis. J. Prod. Anal. 4(1–2):183–210, 1993. doi: 10.1007/BF01073473.
  4. 4.
    Førsund, F. R., Weight restrictions in DEA: Misplaced emphasis? J. Prod. Anal. 40(3):271–283, 2013. doi: 10.1007/s11123-012-0296-9.CrossRefGoogle Scholar
  5. 5.
    Barnum, D., Walton, S., Shields, K., and Schumock, G., Measuring hospital efficiency with data envelopment analysis: Nonsubstitutable vs. substitutable inputs and outputs. J. Med. Syst. 35(6):1393–1401, 2011. doi: 10.1007/s10916-009-9416-0.CrossRefPubMedGoogle Scholar
  6. 6.
    Emrouznejad, A., and Dey, P., Performance measurement in the health sector: Uses of frontier efficiency methodologies and multi-criteria decision making. J. Med. Syst. 35(5):977–979, 2011. doi: 10.1007/s10916-010-9622-9.CrossRefPubMedGoogle Scholar
  7. 7.
    Ramírez-Valdivia, M., Maturana, S., and Salvo-Garrido, S., A multiple stage approach for performance improvement of primary healthcare practice. J. Med. Syst. 35(5):1015–1028, 2011. doi: 10.1007/s10916-010-9438-7.CrossRefPubMedGoogle Scholar
  8. 8.
    Osman, I., Berbary, L., Sidani, Y., Al-Ayoubi, B., and Emrouznejad, A., Data envelopment analysis model for the appraisal and relative performance evaluation of nurses at an intensive care unit. J. Med. Syst. 35(5):1039–1062, 2011. doi: 10.1007/s10916-010-9570-4.CrossRefPubMedGoogle Scholar
  9. 9.
    Kontodimopoulos, N., Papathanasiou, N., Flokou, A., Tountas, Y., and Niakas, D., The impact of non-discretionary factors on DEA and SFA technical efficiency differences. J. Med. Syst. 35(5):981–989, 2011. doi: 10.1007/s10916-010-9521-0.CrossRefPubMedGoogle Scholar
  10. 10.
    Cordero-Ferrera, J., Crespo-Cebada, E., and Murillo-Zamorano, L., Measuring technical efficiency in primary health care: The effect of exogenous variables on results. J. Med. Syst. 35(4):545–554, 2011. doi: 10.1007/s10916-009-9390-6.CrossRefPubMedGoogle Scholar
  11. 11.
    Ketabi, S., Efficiency measurement of cardiac care units of Isfahan hospitals in Iran. J. Med. Syst. 35(2):143–150, 2011. doi: 10.1007/s10916-009-9351-0.CrossRefPubMedGoogle Scholar
  12. 12.
    Barnum, D., Shields, K., Walton, S., and Schumock, G., Improving the efficiency of distributive and clinical services in hospital pharmacy. J. Med. Syst. 35(1):59–70, 2011. doi: 10.1007/s10916-009-9341-2.CrossRefPubMedGoogle Scholar
  13. 13.
    Flokou, A., Kontodimopoulos, N., and Niakas, D., Employing post-DEA cross-evaluation and cluster analysis in a sample of Greek NHS hospitals. J. Med. Syst. 35(5):1001–1014, 2011. doi: 10.1007/s10916-010-9533-9.CrossRefPubMedGoogle Scholar
  14. 14.
    Chuang, C.-L., Chang, P.-C., and Lin, R.-H., An efficiency data envelopment analysis model reinforced by classification and regression tree for hospital performance evaluation. J. Med. Syst. 35(5):1075–1083, 2011. doi: 10.1007/s10916-010-9598-5.CrossRefPubMedGoogle Scholar
  15. 15.
    Rouse, P., Harrison, J., and Turner, N., Cost and performance: Complements for improvement. J. Med. Syst. 35(5):1063–1074, 2011. doi: 10.1007/s10916-010-9520-1.CrossRefPubMedGoogle Scholar
  16. 16.
    Lewis, H., Sexton, T., and Dolan, M., An efficiency-based multicriteria strategic planning model for ambulatory surgery centers. J. Med. Syst. 35(5):1029–1037, 2011. doi: 10.1007/s10916-010-9522-z.CrossRefPubMedGoogle Scholar
  17. 17.
    Blank, J. T., and van Hulst, B., Governance and performance: The performance of Dutch hospitals explained by governance characteristics. J. Med. Syst. 35(5):991–999, 2011. doi: 10.1007/s10916-010-9437-8.PubMedCentralCrossRefPubMedGoogle Scholar
  18. 18.
    Ortiz, J., Meemon, N., Tang, C.-Y., Wan, T. H., and Paek, S., Rural health clinic efficiency and effectiveness: Insight from a nationwide survey. J. Med. Syst. 35(4):671–681, 2011. doi: 10.1007/s10916-009-9404-4.CrossRefPubMedGoogle Scholar
  19. 19.
    Jahangoshai Rezaee, M., Moini, A., and Haji-Ali Asgari, F., Unified performance evaluation of health centers with integrated model of data envelopment analysis and bargaining game. J. Med. Syst. 36(6):3805–3815, 2012. doi: 10.1007/s10916-012-9853-z.CrossRefPubMedGoogle Scholar
  20. 20.
    Chang, D., Chung, J., Sun, K., and Yang, F., A novel approach for evaluating the risk of health care failure modes. J. Med. Syst. 36(6):3967–3974, 2012. doi: 10.1007/s10916-012-9868-5.CrossRefPubMedGoogle Scholar
  21. 21.
    Hadji, B., Meyer, R., Melikeche, S., Escalon, S., and Degoulet, P., Assessing the relationships between hospital resources and activities: A systematic review. J. Med. Syst. 38(10):1–21, 2014. doi: 10.1007/s10916-014-0127-9.CrossRefGoogle Scholar
  22. 22.
    Pelone, F., Kringos, D., Romaniello, A., Archibugi, M., Salsiri, C., and Ricciardi, W., Primary care efficiency measurement using data envelopment analysis: A systematic review. J. Med. Syst. 39(1):1–14, 2014. doi: 10.1007/s10916-014-0156-4.Google Scholar
  23. 23.
    Deidda, M., Lupiáñez-Villanueva, F., Codagnone, C., and Maghiros, I., Using data envelopment analysis to analyse the efficiency of primary care units. J. Med. Syst. 38(10):1–10, 2014. doi: 10.1007/s10916-014-0122-1.CrossRefGoogle Scholar
  24. 24.
    Girginer, N., Köse, T., and Uçkun, N., Efficiency analysis of surgical services by combined use of data envelopment analysis and gray relational analysis. J. Med. Syst. 39(5):1–9, 2015. doi: 10.1007/s10916-015-0238-y.CrossRefGoogle Scholar
  25. 25.
    Rezaee, M. J., and Karimdadi, A., Do geographical locations affect in hospitals performance? A multi-group data envelopment analysis. J. Med. Syst. 2015. doi: 10.1007/s10916-015-0278-3.PubMedGoogle Scholar
  26. 26.
    National_Chicken_Council, Wings 2 rule Superbowl XLIX Meat & Poultry, 2015.Google Scholar
  27. 27.
    Barnum, D., and Gleason, J., Measuring efficiency under fixed proportion technologies. J. Prod. Anal. 35(3):243–262, 2011. doi: 10.1007/s11123-010-0194-y.CrossRefGoogle Scholar
  28. 28.
    Charnes, A., Cooper, W. W., and Rhodes, E., Measuring the efficiency of decision making units. Eur. J. Oper. Res. 2(6):429–444, 1978.CrossRefGoogle Scholar
  29. 29.
    Dyson, R. G., Allen, R., Camanho, A. S., Podinovski, V. V., Sarrico, C. S., and Shale, E. A., Pitfalls and protocols in DEA. Eur. J. Oper. Res. 132(2):245–259, 2001. doi: 10.1016/S0377-2217(00)00149-1.CrossRefGoogle Scholar
  30. 30.
    Cook, W. D., Tone, K., and Zhu, J., Data envelopment analysis: Prior to choosing a model. Omega 44:1–4, 2014. doi: 10.1016/ Scholar
  31. 31.
    Coelli, T. J., Rao, D. S. P., O’Donnell, C. J., and Battese, G. E., An introduction to efficiency and productivity analysis, 2nd edition. Springer, New York, 2005.Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Darold T. Barnum
    • 1
  • Matthew Johnson
    • 2
  • John M. Gleason
    • 3
  1. 1.College of Business Administration, College of PharmacyUniversity of Illinois at ChicagoChicagoUSA
  2. 2.College of MedicineUniversity of Illinois at ChicagoChicagoUSA
  3. 3.College of Business Administration (Emeritus)Creighton UniversityOmahaUSA

Personalised recommendations