Abstract
This paper investigates the pattern of teaching and research performances, the relationship between them, and the convergence for Italian public HEIs in the period 2000–2010, by comparing different bootstrap robust non-parametric frontier estimators. Overall we find an efficiency improvement, mainly driven by research, whereas teaching efficiency increases only in the very first years of the sample period. We also ascertain a slightly positive relationship between research and teaching performances. Furthermore, we find that Italian HEIs converge, in the observed period, although research and teaching do it at a different pace. Our empirical findings are robust to alternative estimators and bootstrapped bias correction.
Similar content being viewed by others
Notes
An increased interest in cross-country comparisons emerged in more recent times, due to the reform of HEI systems within EU (the so-called Bologna Process) that aimed at creating a common European HEI system. An example of such a strand of research is Parteka and Wolszczak-Derlacz (2013) that compares 266 HEIs in 7 European countries in the period 2001–2005.
The advantage of using a non-parametric technique is that it provides a “best performance” frontier that identifies efficient HEIs able to provide a benchmark for under-performing HEIs. An alternative method identifying a frontier is the Stochastic Frontier Analysis (SFA; Aigner et al. 1977; Meeusen and Van den Broeck 1977). In contrast to non-parametric frontier, this approach provides a measure of statistical noise in its estimation of efficiency. However, SFA requires the specification of the functional form of the production process and the distribution of data and error terms.
Notwithstanding their large use, traditional non-parametric models such as DEA and FDH estimators have received some criticism since they rely on extreme points, and they could be extremely sensitive to data selection, aggregation, model specification and data errors (Simar and Wilson 2008). As an alternative, estimators based on partial frontiers and the resulting partial efficiency scores are proposed to provide robust measures of efficiency at extreme data points: Cazals et al. (2002) proposes the nonparametric order-m estimator and Daouia and Simar (2007) order-α quantile type frontiers.
An alternative to analyze the marginal effect of one activity (assumed to be an external or environmental factor) on the other one’s performance is to employ a two-stage DEA analysis (i.e. Sellers-Rubio et al. 2010). A robust semiparametric approach proposed by Simar and Wilson (2007) requires a restrictive separability condition between the input–output space and the space of external or environmental factors. This is certainly unrealistic in our analysis. We thank an anonymous reviewer for this point.
Notice that the ratios are not bounded by 1, in case of m-order estimator.
The sample is coherent with the one used by Agasisti and Wolszczak-Derlacz (2014) and virtually represents all the public HEIs with general purpose in Italian system. Some public HEIs are excluded due the incompleteness of the data in the observed period (namely: Università di Macerata; Politecnico di Bari; Università di Napoli “L’Orientale”; IUAV di Venezia). Other HEIs are excluded from the sample for the peculiarities of those institutions (Scuola Normale Superiore di Pisa; Scuola Superiore di Studi Universitari e Perfezionamento Sant'Anna di Pisa; Scuola Internazionale Superiore di Studi Avanzati di Trieste; Scuola IMT di Lucca; Istituto Universitario di Studi Superiori di Pavia; Università degli Studi di Roma "Foro Italico"; Università per Stranieri di Siena; Università per Stranieri di Perugia).
Results are robust with respect to bias correction and to the assumptions of variable returns to scale and free disposability of inputs, and results are available upon request.
STUD stands for STUD• 104.
GRAD stands for GRAD• 103.
RESEARCH stands for RESEARCH• 103.
To save space we do not report all the results obtained using different estimators. The analytical results of the conditional and unconditional estimates are available upon request.
We also perform a number of other robustness checks to assess effects between other activities and performances including a two-stage robust semiparametric approach proposed by Simar and Wilson (2007) obtaining results comparable with those reported here. Results are available upon request.
However, our findings are robust with respect to variable returns to scale. Estimates are available upon request.
References
Abbott, M., & Doucouliagos, C. (2003). The efficiency of Australian universities: A data envelopment analysis. Economics of Education Review, 22, 89.
Agasisti, T., & Catalano, G. (2006). Governance models of university systems towards quasi-markets? European comparison. Journal of Higher Education policy and Management, 28, 245–262.
Agasisti, T., & Dal Bianco, A. (2009). Reforming the university sector: Effects on teaching efficiency-evidence from Italy. Higher Education, 57, 477–498.
Agasisti, T., & Pérez-Esparrells, C. (2010). Comparing efficiency in a cross-country perspective: The case of Italian and Spanish state universities. Higher Education, 59, 85–103.
Agasisti, T., & Pohl, C. (2012). Comparing German and Italian public universities: Convergence or divergence in the higher education landscape? Managerial and Decision Economics, 33, 71–85.
Agasisti, T., & Wolszczak-Derlacz, J. (2014). Exploring universities’ efficiency differentials between countries in a multi-year perspective: An application of bootstrap DEA and Malmquist index to Italy and Poland, 2001–2011, IRLE working paper no. 113-14.
Aghion, P., Dewatripont, M., Hoxby, C., Mas-Colell, A., & Sapir, A. (2010). The governance and performance of universities: Evidence from Europe and the US. Economic Policy, 25, 7–59.
Aigner, D. J., Lovell, C. A. K., & Schmidt, P. (1977). Formulation and estimation of stochastic frontier production functions. Journal of Econometrics, 6, 21–37.
Archibald, R. B., & Feldman, D. H. (2008). Explaining increases in higher education costs. Journal of Higher Education, 79, 268–295.
Athanassopoulos, A., & Shale, E. (1997). Assessing the comparative efficiency of higher education institutions in the UK by means of data envelopment analysis. Education Economics, 5, 117–134.
Auranen, O., & Nieminenb, M. (2010). University research funding and publication performance—An international comparison. Research Policy, 39, 822–834.
Avkiran, N. K. (2001). Investigating technical and scale efficiencies of Australian universities through data envelopment analysis. Socio-Economic Planning Sciences, 35, 57–80.
Ayadi, R., Boussemart, J., Leleu, H., & Saidane, D. (2013). Mergers and acquisitions in European banking higher productivity or better synergy among business lines? Journal of Productivity Analysis, 39, 165–175.
Badin, L., & Simar, L. (2009). A bias-corrected nonparametric envelopment estimator of frontiers. Econometric Theory, 25, 1289–1318.
Barro, R. J., & Sala-i-Martin, X. (1991). Convergence across states and regions. Brookings Papers Economic Activity, 1, 107–182.
Barro, R. J., & Sala-i-Martin, X. (1992). Convergence. Journal of Political Economy, 100, 223–251.
Beasley, J. (1990). Comparing university departments. Omega, 18, 171–183.
Beasley, J. (1995). Determining teaching and research efficiencies. Journal of the Operational Research Society, 46, 441–452.
Bergantino, A. S., Capozza, C., & Porcelli, F. (2013). Measuring the efficiency of the Italian university system: The role of market structure. A two step DEA analysis at faculty level. Paper presented at the XXV conference of the Italian Association of Public Economics (SIEP).
Bonaccorsi, A., Daraio, C., & Simar, L. (2006). Advanced indicators of productivity of universities an application of robust nonparametric methods to Italian data. Scientometrics, 66, 389–410.
Bonaccorsi, A., Daraio, C., & Simar, L. (2007). Productivity and efficiency of European universities. In A. Bonaccorsi & C. Daraio (Eds.), Universities and strategic knowledge creation: Specialization and performance in Europe. Cheltenham: Edward Elgar.
Brunori, P., Peragine, V., & Serlenga, L. (2012). Fairness in education: The Italian university before and after the reform. Economics of Education Review, 31, 764–777.
Cappellari, L., & Lucifora, C. (2009). The “Bologna Process” and college enrollment decisions. Labour Economics, 16, 638–647.
Casu, B., & Girardone, C. (2010). Integration and efficiency convergence in EU banking markets. Omega, 38, 260–267.
Cazals, C., Florens, J. P., & Simar, L. (2002). Nonparametric frontier estimation: A robust approach. Journal of Econometrics, 106, 1–25.
Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 2, 429–444.
Daouia, A., & Simar, L. (2007). Non-parametric efficiency analysis: A multivariate conditional quantile approach. Journal of Econometrics, 140, 375–400.
Daraio, C., & Simar, L. (2005). Introducing environmental variables in nonparametric frontier models: A probabilistic approach. Journal of Productivity Analysis, 24, 93–121.
Daraio, C., & Simar, L. (2007). Advanced robust and nonparametric methods in efficiency analysis. Methodology and applications. New York: Springer.
De Witte, K., Rogge, N., Cherchye, L., & Van Puyenbroeck, T. (2013a). Accounting for economies of scope in performance evaluations of university professors. Journal of the Operational Research Society, 64, 1595–1606.
De Witte, K., Rogge, N., Cherchye, L., & Van Puyenbroeck, T. (2013b). Economies of scope in research and teaching: A non-parametric investigation. Omega, 41, 305–314.
Debreu, G. (1951). The coefficient of resource utilization. Econometrica, 19, 273–292.
Deprins, D., Simar, L., & Tulkens, H. (1984). Measuring labor efficiency in post offices. In M. Marchand, P. Pestieau, & H. Tulkens (Eds.), The performance of public enterprises: Concepts and measurements. Amsterdam: North Holland.
Di Pietro, G., & Cutillo, A. (2008). Degree flexibility and university drop-out: The Italian experience. Economics of Education Review, 27, 546–555.
Dundar, H., & Lewis, D. (1995). Departmental productivity in American universities: Economies of scale and scope. Economics of Education Review, 14, 119–144.
Fare, R., Grosskopf, S., & Lovell, C. (1994). Production frontiers. Cambridge: Cambridge University Press.
Farrell, M. (1957). The measurement of productive efficiency. Journal of the Royal Statistical Society Series A (General), 120, 253–290.
Feldman, K. (1987). Exploring the relationship between research productivity and college students’ ratings of teachers: Quantitative reviews and the logic of analysis. Paper presented at the Annual Meeting of the American Educational Research Association, Washington, DC.
Flegg, A. T., Allen, D. O., Field, K., & Thurlow, T. W. (2004). Measuring the efficiency of British universities: A multi-period data envelopment analysis. Education Economics, 12, 231–249.
Guccio, C., Martorana, M. F., & Monaco, L. (2015). Evaluating the impact of the Bologna Process on the efficiency convergence of Italian universities: A non-parametric frontier approach. Journal of Productivity Analysis. doi:10.1007/s11123-015-0459.
Hattie, J., & Marsh, H. W. (1996). The relationship between research and teaching: A meta-analysis. Review of Educational Research, 66, 507–542.
Hattie, J., & Marsh, H. W. (2004). One journey to unravel the relationship between research and teaching. Research and teaching: Closing the divide? An international colloquium, Winchester, March 18–19.
Herbst, M. (2007). Financing public universities: The case of performance funding. Dordrecht: Springer.
Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41, 251–261.
Huisman, J., & Van der Wende, M. (2004). The EU and Bologna: Are supra-and international initiatives threatening domestic agendas? European Journal of Education, 39, 349–357.
Johnes, J. (2006). Data envelopment analysis and its applications to the measurement of efficiency in higher education. Economics of Education Review, 25, 273–288.
Johnes, J. (2008). Efficiency and productivity change in the English higher education sector from 1996/97 to 2004/5. Manchester School, 76, 653–674.
Johnes, J., & Johnes, G. (1995). Research funding and performance in U.K. University departments of economics: A frontier analysis. Economics of Education Review, 14, 301–314.
Johnes, J., & Yu, L. (2008). Measuring the research performance of Chinese higher education institutions using data envelopment analysis. China Economic Review, 19, 679–696.
Joumady, O., & Ris, C. (2005). Performance in European higher education: A non-parametric production frontier approach. Education Economics, 13, 189–205.
Kantabutra, S., & Tang, J. (2010). Efficiency analysis of public universities in Thailand. Tertiary Education and Management, 16, 15–33.
Kempkes, G., & Pohl, C. (2010). The efficiency of German universities—Some evidence from nonparametric and parametric methods. Applied Economics, 42, 2063–2079.
Koopmans, T. C. (1951). An analysis of production as an efficient combination of activities. In T. C. Koopmans (Ed.), Activity analysis of production and allocation, Cowles Commission for Research in Economics, monograph 13 (pp. 33–37). New York: Wiley.
Kremer, J. (1990). Construct validity of multiple measures in teaching, research and services and reliability of peer ratings. Journal of Educational Psychology, 82, 213–218.
Kuah, C., & Wong, K. (2011). Efficiency assessment of universities through data envelopment analysis. Procedia Computer Science, 3, 499–506.
Lambert, R., & Butler, N. (2006). The future of European universities: Renaissance or decay?. London: Centre for European Reform.
Laureti, T., Secondi, L., & Biggeri, L. (2014). Measuring the efficiency of teaching activities in Italian universities: An information theoretic approach. Economics of Education Review, 42, 147–164.
Madden, G., Savage, S., & Kemp, S. (1997). Measuring public sector efficiency: A study of economics departments at Australian universities. Education Economics, 5, 153–168.
Meeusen, W., & van den Broeck, J. (1977). Efficiency estimation from Cobb–Douglas production function with composed error. International Economic Review, 8, 435–444.
Neumann, R. (1992). Perceptions of the teaching-research nexus: A framework for analysis. Higher Education, 23, 159–171.
Olivares, M., & Wetzel, H. (2011). Competing in the higher education market: Empirical evidence for economies of scale and scope in German higher education institutions, University of Lüneburg working paper series in economics, no. 223.
Parteka, A., & Wolszczak-Derlacz, J. (2013). Dynamics of productivity in higher education: Cross-European evidence based on bootstrapped Malmquist indices. Journal of Productivity Analysis, 40, 67–82.
Rebora, G., & Turri, M. (2013). The UK and Italian research assessment exercises face to face. Research Policy, 42, 1657–1666.
Sánchez-Barrioluengo, M. (2014a). Articulating the ‘three-missions’ in Spanish universities. Research Policy, 43(10), 1760–1773.
Sav, G. T. (2012). Productivity, efficiency, and managerial performance regress and gains in United States universities: A data envelopment analysis. Advances in Management & Applied Economics, 2, 13–32.
Sellers-Rubio, R., Mas-Ruiz, F. J., & Casado-Dìaz, A. B. (2010). University efficiency: Complementariness versus trade-off between teaching, research and administrative activities. Higher Education Quarterly, 64, 373–391.
Sheather, S. J., & Jones, M. C. (1991). A reliable data-based bandwidth selection method for Kernel density estimation. Journal of the Royal Statistical Society: Series B (Methodological), 53, 683–690.
Shephard, R. W. (1970). Theory of cost and production functions. Princeton: Princeton University Press.
Simar, L., & Wilson, P. (1998). Sensitivity analysis of efficiency scores: How to bootstrap in nonparametric frontier models. Management Science, 44, 49–61.
Simar, L., & Wilson, P. (2000). Statistical inference in nonparametric frontier models: The state of the art. Journal of Productivity Analysis, 13, 49–78.
Simar, L., & Wilson, P. (2002). Nonparametric tests of returns to scale. European Journal of Operations Research, 139, 115–132.
Simar, L., & Wilson, P. (2007). Estimation and inference in two-stage, semi-parametric models of production processes. Journal of Econometrics, 136, 31–64.
Simar, L., & Wilson, P. (2008). Statistical inference in nonparametric frontier models: Recent developments and perspectives. In H. O. Fried, C. A. Knox Lovell, & S. S. Schmidt (Eds.), The measurement of productive efficiency and productivity growth (pp. 421–521). New York: Oxford University Press.
St. Aubyn, M., Pina, A., Garcia, F., & Pais, J. (2009). Study on the efficiency and effectiveness of public spending on tertiary education. Directorate General Economic and Monetary Affairs (DG ECFIN), European Commission economic papers, no. 390.
Tomkins, C., & Green, R. (1988). An experiment in the use of data envelopment analysis for evaluating the efficiency of UK university departments of accounting. Financial Accountability And Management, 4, 147–164.
Wand, M. P., & Jones, M. C. (1995). Kernel smoothing. London: Chapman and Hall Press.
Weill, L. (2009). Convergence in banking efficiency across European countries. Journal of International Financial Markets, Institutions and Money, 19, 818–833.
Wolszczak-Derlacz, J., & Parteka, A. (2010). Scientific productivity of public higher education institutions in Poland—A comparative bibliometric analysis. EY—Better Governance, Warsaw.
Wolszczak-Derlacz, J., & Parteka, A. (2011). Efficiency of European public higher education institutions: A two-stage multicountry approach. Scientometrics, 89, 887–917.
Worthington, A. (2001). An empirical survey of frontier efficiency measurement techniques in education. Education Economics, 9, 245–268.
Zhang, T., & Matthews, K. (2012). Efficiency convergence properties of Indonesian banks 1992–2007. Applied Financial Economics, 22, 1465–1478.
Acknowledgments
Preliminary versions of this paper were presented in the XXVII annual congress of the Italian Society of Public Economics (Ferrara, 2015), and seminars at the Kyoto Sangyo University and University of Munich. We thank the participants and discussants, as well as two anonymous reviewers for several helpful comments. The usual caveat applies.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Guccio, C., Martorana, M.F. & Mazza, I. Efficiency assessment and convergence in teaching and research in Italian public universities. Scientometrics 107, 1063–1094 (2016). https://doi.org/10.1007/s11192-016-1903-8
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-016-1903-8