Abstract
New Public Management helps universities and research institutions to perform in a highly competitive research environment. Evaluating publicly financed research improves transparency, helps in reflection and self-assessment, and provides information for strategic decision making. In this paper we provide empirical evidence using data from a Collaborative Research Center (CRC) on financial inputs and research output from 2005 to 2016. After selecting performance indicators suitable for a CRC, we describe main properties of the data using visualization techniques. To study the relationship between the dimensions of research performance, we use a time fixed effects panel data model and fixed effects Poisson model. With the help of year dummy variables, we show how the pattern of research productivity changes over time after controlling for staff and travel costs. The joint depiction of the time fixed effects and the research project’s life cycle allows a better understanding of the development of the number of discussion papers over time.
Similar content being viewed by others
References
Abramo, G., D’Angelo, C. A., & Di Costa, F. (2009). Research collaboration and productivity: Is there correlation? Higher Education, 57(2), 155–171.
Abramo, G., & D’Angelo, C. A. (2011). National-scale research performance assessment at the individual level. Scientometrics, 86, 347–364.
Barra, C., & Zotti, R. (2016). Measuring efficiency in higher education: An empirical study using a bootstrapped data envelopment analysis. International Advances in Economic Research, 22, 11–33.
Bolli, T., & Somogyi, F. (2011). Do competitively acquired funds induce universities to increase productivity? Research Policy, 40(1), 136–147.
Bornmann, L. (2013). How to analyze percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes, and top-cited papers. Journal of the American Society for Information Science and Technology, 64(3), 587–595.
Cameron, A. C., & Miller, D. L. (2015). A practitioner’s guide to cluster–robust inference. Journal of Human Resources, 50(2), 317–373.
Carayol, N., & Matt, M. (2004). Does research organization influence academic production? Laboratory level evidence from a large European university. Research Policy, 33(8), 1081–1102.
CRC 649. (2016). http://sfb649.wiwi.hu-berlin.de/about/index.php. Accessed 11 January 2018.
CRC Project on GitHub. http://www.github.com/QuantLet/CRC. Accessed 11 January 2018.
Deutsche Forschungsgemeinschaft (DFG, eng. - German Research Foundation). (2017). Chancengleichheits-Monitoring 2017. Antragstellung und -erfolg von Wissenschaftlerinnen bei der DFG. http://www.dfg.de/download/pdf/foerderung/grundlagen_dfg_foerderung/chancengleichheit/chancengleichheits_monitoring_2017.pdf. Accessed 01 August 2018.
Deutsche Forschungsgemeinschaft (DFG, eng. - German Research Foundation). (2018). http://www.dfg.de/en//research_funding/programmes/coordinated_programmes/collaborative_research_centres/index.html. Accessed 11 January 2018.
Diem, A., & Wolter, S. C. (2013). The use of bibliometrics to measure research performance in educational sciences. Research in Higher Education, 54, 86–114.
Donner, P., & Aman, V. (2015). Quantilbasierte Indikatoren für impact und Publikationsstartegie. Ergebnisse für Deutschland in allen Fachdisziplinen in den Jahren 2000 bis 2011, (p. 8). IFQ: Studien zum deutschen Innovationssystem.
Grözinger, G. & Leusing, B. (2006). Wissenschaftsindikatoren an Hochschulen. Europa-Universität Flensburg, International Institute of Management, Discussion Papers. 012, https://EconPapers.repec.org/RePEc:fln:wpaper:012.
Hausman, J., Hall, B. H., & Griliches, Z. (1984). Econometric models for count data with an application to the patents-R & D relationship. Econometrica, 52(4), 909–938.
Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.
Hood, C. (1991). A public management for all seasons? Public Administration, 69(1), 3–19.
Hornbostel, S. (1991). Drittmitteleinverbung. Ein Indikator für universitäre Forschungsleistungen. Beiträge zu Hochschulforschung, 1, 57–84.
Jansen, D., Wald, A., Franke, K. et al. (2007). Drittmittel als Performanzindikator der Wissenschaftlichen Forschung. Koelner Z. Soziol. u. Soz. Psychol., 59(1), 125–149.
JEL (Journal of Economic Literature) Classification System. (2018). https://www.aeaweb.org/econlit/jelCodes.php?view=jel. Accessed 11 January 2018.
Lee, G. J. (2010). Assessing publication performance of research units: Extensions through operational research and economic techniques. Scientometrics, 84(3), 717–734.
Moed, H. F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 265–277.
Moed, H. F., de Moya-Anegón, F., López-Illescas, C., & Visser, M. (2011). Is concentration of university research associated with better research performance? Journal of Informetrics, 5, 649–658.
Pastor, J. M., & Serrano, L. (2016). The determinants of the research output of universities: Specialization, quality and inefficiencies. Scientometrics, 1029(2), 1255–1281.
Pastor, J. M., Serrano, L., & Zaera, I. (2015). The research output of European higher education institutions. Scientometrics, 102(3), 1867–1893.
Schmoch, U., & Schubert, T. (2009). Sustainability of incentives for excellent research—The German case. Scientometrics, 81(1), 195–218.
Schröder, S., Welter, F., Leisten, I., Richert, A., & Jeschke, S. (2014). Research performance and evaluation? Empirical results from collaborative research centers and clusters of excellence in Germany. Research Evaluation, 23(3), 221–232.
Stock, J. H., & Watson, M. W. (2003). Introduction to econometrics (1st ed.). London: Pearson.
Van den Berghe, H., Houben, J. A., de Bruin, R. E., Moed, H. F., Kint, A., Luwel, M., et al. (1998). Bibliometric indicators of university research performance in Flanders. Journal of the American Society for Information Science, 49(1), 59–67.
Wissenschaftsrat. (2004). Empfehlungen zu Rankings im Wissenschaftssystem Teil 1: Forschung (pp. 6285–04). Drs: Hamburg.
Wissenschaftsrat. (2011). Empfehlungen zur Bewertung und Steuerung von Forschungsleistung, Drs 1656–11.
Wissenschaftsrat. (2012). Bericht der Steuerungsgruppe zur Pilotstudie zur Weiterentwicklung des Forschungsratings (pp. 2815–12). Drs: Köln.
Wooldridge, J. M. (1999). Distribution-free estimation of some nonlinear panel data models. Journal of Econometrics, 90, 77–97.
Wooldridge, J. M. (2002). Econometric analysis of cross section and panel data (1st ed.). Cambridge: MIT Press Books.
Wooldridge, J. M. (2016). Introductory econometrics: A modern approach (6th ed.). Boston: Cengage Learning.
Zharova, A., Härdle, W.K. & Lessmann, S. (2017). Is scientific performance a function of funds? SFB 649 Discussion Paper, 2017(28).
Zheng, J., & Liu, N. (2015). Mapping of important international academic awards. Scientometrics, 104(3), 763–791.
Author information
Authors and Affiliations
Corresponding author
Additional information
Financial support from the German Research Foundation (DFG) via Collaborative Research Center 649 “Economic Risk” and International Research Training Group 1792 “High Dimensional Nonstationary Time Series”, Humboldt-Universität zu Berlin, is gratefully acknowledged. We are thankful for the assistance provided by Nicole Hermann und Dominik Prugger.
Rights and permissions
About this article
Cite this article
Zharova, A., Tellinger-Rice, J. & Härdle, W.K. How to measure the performance of a Collaborative Research Center. Scientometrics 117, 1023–1040 (2018). https://doi.org/10.1007/s11192-018-2910-8
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-018-2910-8