Abstract
The analysis of productivity in Higher Education Institutions (HEIs) at a European level reveals enormous differences in output per researcher across countries. This study develops a 5-step methodology that explicitly considers the quality of scientific output in EU universities and its specialisations to explain and decompose the differences in output per university researcher in terms of (a) differences in efficiency within each field of science (FOS), (b) differences in FOS specialisation of HEIs in each country, (c) differences in quality, and (d) differences in allocation of resources per researcher. The inefficiency levels estimated show that across the EU as a whole there is a substantial margin for increasing research output without having to spend more resources. There are also major differences between countries in terms of inefficiency. The main sources of heterogeneity in scientific output in the HEIs of the EU are the differences in resources allocated per researcher and, to a lesser extent, the differences in efficiency within each knowledge field. The differences in quality and in specialisation also play a smaller role in determining differences in output.
Similar content being viewed by others
Notes
Some studies propose the additional use of diverse indicators of the quality of university teaching, such as the drop-out rate, the performance rate, the student-teacher ratio, expenditure per student, the number of information technology (IT) and library staff per student, expenditure per student, etc. See Pérez et al. (2015a, b).
Data provided by SCIMAGO Journal & Country Rank refer to the total number of scientific publications produces by a country. 99 % of the EU-28’s scientific output comes from universities (64.3 %), Public research centres (22.8 %) and Hospitals (11.8 %). For this reason the data on patents, publications, citations, R&D expenditure and R&D personnel provided throughout this paper refer to Higher Education (universities) and Government sector (Public Research Centres and Hospitals) as a whole.
The information is available on the following website: http://www.scimagojr.com/countryrank.php
R&D expenditures will cover also expenditures for R&D personnel. The data do not allow to disentangle personnel expenditure from other R&D expenditure by FOS. If we want to take into account the specialization effect, as we do, we have to use total R&D expenditures as input (which include both non-personnel R&D expenditure and research wages, in fact a measure of abundance of resources for researchers). It is also important to consider research as an output coming from more than one input. Labour is a very important input and other types of R&D expenditure are also very relevant. HEIs have the option of allocating more resources to their researchers, employing better qualified and paid researchers or using more researchers. We think that this is an important fact that needs to be considered. It is possible to disentangle personnel expenditure from other R&D expenditure but only without taking into account FOS specialization. We have carried out that exercise using two inputs: only the non-personnel R&D expenditure and R&D personnel. This analysis avoids any potential issue of double accounting. The coefficients of correlation between the inefficiency indicators obtained from this new exercise and the comparable inefficiency indicators obtained and discussed in the paper range between 0.96 and 0.98. Therefore, the results are maintained, showing robustness to this potential issue.
The positive impacts of universities on the economic growth of their countries’ economies have been widely demonstrated in the literature, especially in the case of North American universities (Pastor et al. 2013).
Alternatively, Farrel also proposed to measure efficiency from the perspective of the potential reduction of inputs given a vector of outputs (Farrell 1957).
Maudos et al. (2000) use a similar methodology to analyse the regional output differences.
References
Abbott, M., & Doucouliagos, C. (2003). The efficiency of Australian universities: A data envelopment analysis. Economics of Education Review, 22(1), 89–97.
Abbott, M., & Doucouliagos, H. (2004). Research output of Australian universities. Education Economics, 12(3), 251–265.
Agrawal, A. K., & Henderson, R. M. (2002). Putting patents in context: Exploring knowledge transfer from MIT. Management Science, 48, 44–60.
Azoulay, P., Ding, W., & Stuart, T. (2009). The impact of academic patenting on the rate, quality and direction of (public) research output. The Journal of Industrial Economics, 57, 637–676.
Bonaccorsi, A., & Daraio, C. (2007). Universities and strategic knowledge creation: Specialization and performance in Europe. Cheltenham: Edward Elgar Publishing.
Breschi, L., Lissoni, F., & Montobbio, F. (2007). The scientific productivity of academic inventors: New evidence from Italian data. Economics of Innovation and New Technology, 16(2), 101–118.
Buenstorf, (2009). Is commercialization good or bad for science? Individual-level evidence from the Max Planck Society. Research Policy, 38(2), 281–292.
Carayol, N. (2007). Academic incentives, research organization and patenting at a large French University. Economics of Innovation and New Technology, 16(2), 119–138.
Center for Science and Technology Studies (CWTS). (2009). The Leiden ranking. Retrieved November from http://www.cwts.nl/ranking/LeidenRankingWebSite.html.
Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision-making units. European Journal of Operational Research, 2, 429–444. doi:10.1016/0377-2217(78)90138-8.
Crespi, G., D’Este, P., Fontana, R., & Geuna, A. (2011). The impact of academic patenting on university research and its transfer. Research Policy, 40, 55–68.
de Groot, H., McMahon, W., & Volkwein, J. F. (1991). The cost structure of American Research Universities. The Review of Economics and Statistics, 73(3), 424–431.
Fabrizio, K., & DiMinin, A. (2008). Commercializing the laboratory: The relationship between faculty patenting and publishing. Research Policy, 37(5), 914–931.
Falagas, M. E., Kouranos, V. D., Arencibia-Jorge, R., & Karageorgopoulos, D. E. (2008). Comparison of SCImago journal rank indicator with journal impact factor. The FASEB Journal, 22(22), 2623–2628.
Farrell, M. (1957). The measurement of productive efficiency. Journal of the Royal Statistics Society, Series A, 120(3), 253–281.
Filippini, M., & Lepori, B. (2007). Cost structure, economies of capacity utilization and scope in Swiss higher education institutions. In Universities and strategic knowledge creation: Specialization and performance in Europe (pp. 272–304).
Flegg, A. T., Allen, D. O., Field, K., & Thurlow, T. W. (2004). Measuring the efficiency of British universities: A multi-period data envelopment analysis. Education Economics, 12(3), 231–249.
Glass, J. C., McKillop, D. G., & Hyndman, N. (1995). Efficiency in the provision of university teaching and research: An empirical analysis of UK universities. Journal of Applied Econometrics, 10(1), 61–72.
González-Albo, B., Moreno, L., Morillo, F., & Bordons, M. (2012). Bibliometric indicators for the analysis of the research performance of a multidisciplinary institution: The CSIC. Revista Española de Documentación Científica, 35(1), 9–37.
Guccio, C., Martorana, M. F., & Monaco, L. (2016). Evaluating the impact of the Bologna Process on the efficiency convergence of Italian universities: A non-parametric frontier approach. Journal of Productivity Analysis, 45(3), 275–298.
Johnes, G. (1988). Determinants of research output in economics departments in British universities. Research Policy, 17(3), 171–178.
Johnes, J. (1996). Performance assessment in higher education in Britain. European Journal of Operational Research, 89(1), 18–33.
Johnes, J. (2006). Data envelopment analysis and its application to the measurement of efficiency in higher education. Economics of Education Review, 25(3), 273–288.
Johnes, G., & Johnes, J. (2009). Higher education institutions’ costs and efficiency: Taking the decomposition a further step. Economics of Education Review, 28(1), 107–113. doi:10.1016/j.econedurev.2008.02.001.
Kempkes, G., & Pohl, C. (2010). The efficiency of German universities—some evidence from nonparametric and parametric methods. Applied Economics, 42(16), 2063–2079.
Klitkou, A., & Gulbrandsen, M. (2010). The relationship between academic patenting and scientific publishing in Norway. Scientometrics, 82(1), 93–108.
Kuah, C. T., & Wong, K. (2011). Efficiency assessment of universities through data envelopment analysis. Procedia Computer Science, 3, 499–506.
Lepori, B. (2007). Paterns in the Swiss higher education system. In Universities and strategic knowledge creation: Specialization and performance in Europe (pp. 209–240).
Maudos, J., Pastor, J. M., & Serrano, L. (2000). Efficiency and productive specialization: An application to the Spanish regions. Regional Studies, 34(9), 829–842.
Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht: Springer.
Nazarko, J., & Šaparauskas, J. (2014). Application of DEA method in efficiency evaluation of public higher education institutions. Technological and Economic Development of Economy, 20(1), 25–44.
Pastor, J. M., Peraita, C., & Pérez, F. (2015a). Estimating the long-term economic impacts of Spanish universities on the National Economy. Papers in Regional Science,. doi:10.1111/pirs.12157.
Pastor, J. M., Pérez, F., & Fernández de Guevara, J. (2013). Measuring the local economic impact of universities: An approach that considers uncertainty. Higher Education, 65(5), 539–564.
Pastor, J. M., Serrano, L., & Zaera, I. (2015b). The research output of European higher education institutions. Scientometrics, 102(3), 1867–1893.
Rey, O. (2009). Quality indicators and educational research publications: Which publications count?. Dossier d’actualité No. 46–June–July. Service de Veille scientifique et technologique, L’Institut Français de l’Éducation. http://ife.ens-lyon.fr/vst/DA/detailsDossier.php?parent=accueil&dossier=46&lang=en.
Salas, V. (2012). La producción universitaria. El caso chileno. Departamento de Economía. Universidad de Santiago de Chile.
Sarrico, C. S., & Dyson, R. G. (2004). Restricting virtual weights in data envelopment analysis. European Journal of Operational Research, 159(1), 17–34.
Sarrico, C. S., Teixeira, P., Rosa, M. J., & Cardoso, M. F. (2009). Subject mix and productivity in Portuguese universities. European Journal of Operational Research, 197, 287–295.
Schubert, T., & Kroll, H. (2014). Universities’ effects on regional GDP and unemployment: The case of Germany. Papers in Regional Science,. doi:10.1111/pirs.12150.
SCImago. (2012). SIR world report 2012. Global ranking. SCIMAGO Institutions Rankings (SIR). http://www.scimagoir.com.
SCImago. (2012). SJR—SCImago Journal & Country Rank. Retrieved July 2012, from http://www.scimagojr.com.
Stephan, P. E., Gurmu, S., Sumell, A. J., & Black, G. (2007). Who’s patenting in the university? Evidence from the survey of doctorate recipients. Economics of Innovation and New Technology, 16(2), 71–99.
Thanassoulis, E., Kortelainen, M., Johnes, G., & Johnes, J. (2011). Costs and efficiency of higher education institutions in England: A DEA analysis. Journal of the Operational Research Society, 62(7), 1282–1297. doi:10.1057/jors.2010.68.
Vieira, E. S., & Gomes, J. A. N. F. (2009). A comparison of scopus and web of science for a typical university. Scientometrics, 81(2), 587–600.
Worthington, A. C., & Lee, B. L. (2008). Efficiency, technology and productivity change in Australian universities, 1998–2003. Economics of Education Review, 27(3), 285–298.
Acknowledgments
The authors would like to thank a reviewer for the comments that helped to improve an earlier version of the manuscript. This paper was developed as part of the SPINTAN project funded by the European Commission. This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under Grant Agreement No: 612774. The authors are grateful for funding from Research Project ECO2015-70632-R (Ministry of the Economy and Competitiveness).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pastor, J.M., Serrano, L. The determinants of the research output of universities: specialization, quality and inefficiencies. Scientometrics 109, 1255–1281 (2016). https://doi.org/10.1007/s11192-016-2102-3
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-016-2102-3