article

The impact of research funding on scientific outputs: Evidence from six smaller European countries

Authors:

Philip ShapiraAuthors Info & Claims

Journal of the Association for Information Science and Technology, Volume 67, Issue 3

Pages 715 - 730

https://doi.org/10.1002/asi.23406

Published: 01 March 2016 Publication History

Abstract

We investigate the relationships between the citation impacts of scientific papers and the sources of funding that are acknowledged as having supported those publications. We examine several relationships potentially associated with funding, including first citation, total citations, and the chances of becoming highly cited. Furthermore, we explore the links between citations and types of funding by organization and also with combined measures of funding. In particular, we examine the relationship between funding intensity and funding variety and citation. Our empirical work focuses on six small advanced European economies, applying a zero inflated negative binomial model to a set of more than 240,000 papers authored by researchers from these countries. We find that funding is not related to the first citation but is significantly related to the number of citations and top percentile citation impact. Additionally, we find that citation impact is positively related to funding variety and negatively related with funding intensity. Finally there is an inverse relationship between the relative frequency of funding and citation impact. The results presented in the paper provide insights for the design of research programs and the structure of research funding and for the behavior and strategies of research and sponsoring organizations.

References

[1]

Acosta, M., Coronado, D., Ferrándiz, E., &León, M.D. 2011. Factors affecting inter-regional academic scientific collaboration within Europe: The role of economic distance. Scientometrics, Volume 87 Issue 1, pp.63-74.

Digital Library

[2]

Arnold, E. 2012. Understanding long-term impacts of R&D funding: The EU framework programme. Research Evaluation, Volume 21 Issue 5, pp.332-343.

[3]

Arnold, J.G., &Fohrer, N. 2005. SWAT2000: Current capabilities and research opportunities in applied watershed modelling. Hydrological Processes, Volume 19 Issue 3, pp.563-572.

[4]

Baldi, S. 1998. Normative versus social constructivist processes in the allocation of citations: A network-analytic model. American Sociological Review, Volume 63 Issue 6, pp.829-846.

[5]

Bayona-Saez, C., &Garcia-Marco, T. 2010. Assessing the effectiveness of the Eureka Program. Research Policy, Volume 39 Issue 10, pp.1375-1386.

[6]

Bleda, M., &<familyNamePrefix>del</familyNamePrefix>Rio, P. 2013. The market failure and the systemic failure rationales in technological innovation systems. Research Policy, Volume 42 Issue 5, pp.1039-1052.

[7]

Cassi, L., Corrocher, N., Malerba, F., &Vonortas, N. 2008. The impact of EU-funded research networks on knowledge diffusion at the regional level. Research Evaluation, Volume 17 Issue 4, pp.283-293.

[8]

Chen, C. 2012. Predictive effects of structural variation on citation counts. Journal of the American Society for Information Science and Technology, Volume 63 Issue 3, pp.431-449.

Digital Library

[9]

Clarysse, B., Tartari, V., &Salter, A. 2011. The impact of entrepreneurial capacity, experience and organizational support on academic entrepreneurship. Research Policy, Volume 40 Issue 8, pp.1084-1093.

[10]

Costas, R., &<familyNamePrefix>van</familyNamePrefix>Leeuwen, T.N. 2012. Approaching the "reward triangle": General analysis of the presence of funding acknowledgments and "peer interactive communication" in scientific publications. Journal of the American Society for Information Science and Technology, Volume 63 Issue 8, pp.1647-1661.

Digital Library

[11]

Department for Business, Innovation & Skills</publisherName>. 2011. Funding for EU research and innovation from 2014: A UK perspective. London, United Kingdom: <publisherName>Author. Retrieved from "https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/32484/11-901-funding-eu-research-innovation-from-2014.pdf"

[12]

Desmarais, B.A., &Harden, J.J. 2013. Testing for zero inflation in count models: Bias correction for the Vuong test. Stata Journal, Volume 13 Issue 4, pp.810-835.

[13]

Didegah, F., &Thelwall, M. 2013. Determinants of research citation impact in nanoscience and nanotechnology. Journal of the American Society for Information Science and Technology, Volume 64 Issue 5, pp.1055-1064.

[14]

Engineering and Physical Sciences Research Council</publisherName>. 2014. Funding guide. Swindon, United Kingdom: <publisherName>Author. Retrieved from "http://www.epsrc.ac.uk/files/funding/how-to-apply/fundingroutes/funding-guide/"

[15]

Etzkowitz, H., &Leydesdorff, L.A. 1997. Universities and the global knowledge economy: A triple helix of University-Industry-Government relations. London: Continuum International Publishing Group, Limited.

[16]

European Research Council. 2012. Guide for ERC grant holders. Brussels, Belgium: European Research Council Executive Agency. Retrieved from "ftp://ftp.cordis.europa.eu/pub/fp7/docs/erc-guide-grantholders_en.pdf". Final Version, July 2012. Ref. Ares2012894501-23/07/2012.

[17]

European Union.2012. New practical guide to EU funding opportunities for research and innovation. Annexes, Luxembourg: European Union, CORDIS Publications Office. Retrieved from "http://cordis.europa.eu/eu-funding-guide/annex04_en.html#annexes"

[18]

Georghiou, L.G., &Metcalfe, J.S. 1993. Evaluation of the impact of European community research programs upon industrial competitiveness. R&D Management, Volume 23 Issue 2, pp.161-169.

[19]

Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., &Trow, M. 1994. The new production of knowledge: The dynamics of science and research in contemporary societies. Thousand Oaks, CA: Sage.

[20]

Greene, W.H. 2003. Econometric analysis. New York: Pearson Education.

[21]

Grupp, H., &Mogee, M.E. 2004. Indicators for national science and technology policy: How robust are composite indicators? Research Policy, Volume 33 Issue 9, pp.1373-1384.

[22]

Haslam, N., Ban, L., Kaufmann, L., Loughnan, S., Peters, K., Whelan, J., &Wilson, S. 2008. What makes an article influential? Predicting impact in social and personality psychology. Scientometrics, Volume 76 Issue 1, pp.169-185.

[23]

Heinze, T., Shapira, P., Rogers, J.D., &Senker, J.M. 2009. Organizational and institutional influences on creativity in scientific research. Research Policy, Volume 38 Issue 4, pp.610-623.

[24]

Jennings, M. 2012. Academic output from EU-funded health research projects. The Lancet, Volume 380 Issue 9857, pp.1903.

[25]

Jump, P. 2014. Switzerland downgraded by EU on research involvement. Times Higher Education. Retrieved from "http://www.timeshighereducation.co.uk/news/switzerland-downgraded-by-eu-on-research-involvement/2011691.article" 26 February 2014

[26]

Katz, J.S., &Hicks, D. 1997. How much is a collaboration worth? A calibrated bibliometric model. Scientometrics, Volume 40 Issue 3, pp.541-554.

[27]

Katz, J.S., &Martin, B.R. 1997. What is research collaboration? Research Policy, Volume 26 Issue 1, pp.1-18.

[28]

Kleijn, D., &Sutherland, W.J. 2003. How effective are European agri-environment schemes in conserving and promoting biodiversity? Journal of Applied Ecology, Volume 40 Issue 6, pp.947-969.

[29]

Kolodny, H., Stymne, B., Shani, R., Figuera, J.R., &Lillrank, P. 2001. Design and policy choices for technology extension organizations. Research Policy, Volume 30 Issue 2, pp.201-225.

[30]

Kostoff, R.N. 1993. Semiquantitative methods for research impact assessment. Technological Forecasting and Social Change, Volume 44 Issue 3, pp.231-244.

[31]

Kreilkamp, K. 1971. Hindsight and the real world of science policy. Science Studies, Volume 1 Issue 1, pp.43-66.

[32]

Laredo, P. 1998. The networks promoted by the framework programme and the questions they raise about its formulation and implementation. Research Policy, Volume 27 Issue 6, pp.589-598.

[33]

Lee, Y.-G. 2010. Sectoral strategic differences of technological development between electronics and chemistry: A historical view from analyses of Korean-invented US patents during the period of 1989-1992. Scientometrics, Volume 82 Issue 1, pp.83-92.

[34]

Lee, Y.-G., Lee, J.-D., Song, Y.-I., &Lee, S.-J. 2007. An in-depth empirical analysis of patent citation counts using zero-inflated count data model: The case of KIST. Scientometrics, Volume 70 Issue 1, pp.27-39.

[35]

Lewison, G. 1994. Publications from the European-Community Biotechnology Action Program BAP-Multinationality, acknowledgment of support, and citations. Scientometrics, Volume 31 Issue 2, pp.125-142.

[36]

Lewison, G. 1998. Gastroenterology research in the United Kingdom: Funding sources and impact. Gut, Volume 43 Issue 2, pp.288-293.

[37]

Lewison, G. 2003. Beyond outputs: New measures of biomedical research impact. Aslib Proceedings, Volume 55 Issue 1-2, pp.32-42.

[38]

Lewison, G. 2009. Financial acknowledgements on the Web of science: A new resource for bibliometric analysis. In B.Larsen &J.Leta Eds., Proceedings of Issi 2009-12th International Conference of the International Society for Scientometrics and Informetrics Vol. Volume 2, pp. pp.968-969. Leuven, Belgium: Int Soc Scientometrics & Informetrics-Issi.

[39]

Lewison, G., &Dawson, G. 1998. The effect of funding on the outputs of biomedical research. Scientometrics, Volume 41 Issue 1-2, pp.17-27.

[40]

Lewison, G., &Markusova, V. 2010. The evaluation of Russian cancer research. Research Evaluation, Volume 19 Issue 2, pp.129-144.

[41]

Lewison, G., Grant, J., &Jansen, P. 2001. International gastroenterology research: Subject areas, impact, and funding. Gut, Volume 49 Issue 2, pp.295-302.

[42]

Long, J.S. 1997. Regression models for categorical and limited dependent variables. Thousand Oaks, CA: Sage.

[43]

Luukkonen, T. 2012. Conservatism and risk-taking in peer review: Emerging ERC practices. Research Evaluation, Volume 21 Issue 1, pp.48-60.

[44]

Marin, P.L., &Siotis, G. 2008. Public policies towards research joint venture: Institutional design and participants' characteristics. Research Policy, Volume 37 Issue 6-7, pp.1057-1065.

[45]

Martin, S., &Scott, J.T. 2000. The nature of innovation market failure and the design of public support for private innovation. Research Policy, Volume 29 Issue 4-5, pp.437-447.

[46]

Messinis, G. 2011. Triadic citations, country biases and patent value: The case of pharmaceuticals. Scientometrics, Volume 89 Issue 3, pp.813-833.

Digital Library

[47]

Mowery, D.C., Nelson, R.R., &Martin, B.R. 2010. Technology policy and global warming: Why new policy models are needed or why putting new wine in old bottles won't work. Research Policy, Volume 39 Issue 8, pp.1011-1023.

[48]

Mytelka, L.K., &Smith, K. 2002. Policy learning and innovation theory: An interactive and co-evolving process. Research Policy, Volume 31 Issue 8-9, pp.1467-1479.

[49]

Nemet, G.F. 2009. Demand-pull, technology-push, and government-led incentives for non-incremental technical change. Research Policy, Volume 38 Issue 5, pp.700-709.

[50]

Neufeld, J., Huber, N., &Wegner, A. 2013. Peer review-based selection decisions in individual research funding, applicants' publication strategies and performance: The case of the ERC Starting Grants. Research Evaluation, Volume 22 Issue 4, pp.237-247.

[51]

Noyons, E., Teigland, R., Schenkel, A., &Maier, J. 2009. Evaluating a research funding program: Measuring the impact of EU 6th Framework Programme, Conference of the International Society for Scientometrics and Informatics ISSI, Rio de Janeiro, Brazil.

[52]

National Science Foundation. 2001. Grant proposal guide. Arlington, VA: Author. Retrieved from "http://www.nsf.gov/pubs/2002/nsf022/nsf022.pdf". NSF 02-2.

[53]

Porter, A.L., &Rafols, I. 2009. Is science becoming more interdisciplinary? Measuring and mapping six research fields over time. Scientometrics, Volume 81 Issue 3, pp.719-745.

[54]

Rafols, I., Porter, A.L., &Leydesdorff, L. 2010. Science overlay maps: A new tool for research policy and library management. Journal of the American Society for Information Science and Technology, Volume 61 Issue 9, pp.1871-1887.

Digital Library

[55]

Rigby, J. 2009. Comparing the scientific quality achieved by funding instruments for single grant holders and for collaborative networks within a research system: Some observations. Scientometrics, Volume 78 Issue 1, pp.145-164.

[56]

Rigby, J. 2011. Systematic grant and funding body acknowledgement data for publications: New dimensions and new controversies for research policy and evaluation. Research Evaluation, Volume 20 Issue 5, pp.365-375.

[57]

Rigby, J. 2013. Looking for the impact of peer review: Does count of funding acknowledgements really predict research impact? Scientometrics, Volume 94 Issue 1, pp.57-73.

Digital Library

[58]

Rigby, J., &Julian, K. 2013. Optimizing research impact by allocating funding to researcher grant portfolios: some evidence on a policy option. Paper presented at the International Society for Scientometrics and Informetrics ISSI 2013, Vienna, Austria.

[59]

Scherngell, T., Roche, I., Horlesberger, M., Besagni, D., Zuger, M.E., &Holste, D. 2013. Initial comparative analysis of model and peer review process for ERC starting grant proposals. Research Evaluation, Volume 22 Issue 4, pp.248-257.

[60]

Shapira, P., &Wang, J. 2010. Follow the money. Nature, Volume 468 Issue 7324, pp.627-628.

[61]

Sherwin, C.W., &Isenson, R.S. 1967. Project hindsight. Science, Volume 156 Issue 3782, pp.1571-1577.

[62]

Sirtes, D. 2013. Funding acknowledgements for the German Research Foundation DFG. The dirty data of the web of science database and how to clean it up. Paper presented at the International Society for Scientometrics and Informetrics ISSI 2013, Vienna, Austria.

[63]

StataCorp. 2011. Stata 12 base reference manual. College Station, TX: Stata Press.

[64]

Stephan, P. 2012. Perverse incentives. Nature, Volume 484 Issue 7392, pp.29-31.

[65]

Thomas, D., &Nedeva, M. 2012. Characterizing researchers to study research funding agency impacts: The case of the European Research Council's Starting Grants. Research Evaluation, Volume 21 Issue 4, pp.257-269.

[66]

Upham, S.P., Rosenkopf, L., &Ungar, L.H. 2010. Positioning knowledge: Schools of thought and new knowledge creation. Scientometrics, Volume 83 Issue 2, pp.555-581.

[67]

<familyNamePrefix>van den</familyNamePrefix>Besselaar, P., &Sandstrom, U. 2013. The effect of funding modes on the quality of knowledge production. Paper presented at the International Society for Scientometrics and Informetrics ISSI 2013, Vienna, Austria.

[68]

Vuong, Q.H. 1989. Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica: Journal of the Econometric Society, Volume 57 Issue 2, pp.307-333.

[69]

Wang J, Shapira P 2015 Is There a Relationship between Research Sponsorship and Publication Impact? An Analysis of Funding Acknowledgments in Nanotechnology Papers. PLoS ONE Volume 10 Issue 2: pp.e0117727.

[70]

Wang, X.W., Liu, D., Ding, K., &Wang, X.R. 2012. Science funding and research output: A study on 10 countries. Scientometrics, Volume 91 Issue 2, pp.591-599.

Digital Library

[71]

Whitley, R. 2000. The intellectual and social organization of the sciences 2nd ed. Oxford, UK: Oxford University Press.

[72]

Yegros-Yegros, A., &Costas, R. 2013. Analysis of the web of science funding acknowledgement information for the design of indicators on "external funding attraction." Paper presented at the International Society for Scientometrics and Informetrics ISSI 2013, Vienna, Austria.

[73]

Yoshikane, F. 2013. Multiple regression analysis of a patent's citation frequency and quantitative characteristics: The case of Japanese patents. Scientometrics, Volume 96 Issue 1, pp.365-379.

Digital Library

Cited By

El-Ouahi J(2024)Research funding in the Middle East and North Africa: analyses of acknowledgments in scientific publications indexed in the Web of Science (2008–2021)Scientometrics10.1007/s11192-024-04983-8129:6(2933-2968)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s11192-024-04983-8
Li MWang Y(2024)Influence of political tensions on scientific productivity, citation impact, and knowledge combinationsScientometrics10.1007/s11192-024-04973-w129:4(2337-2370)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s11192-024-04973-w
van der Pol J(2024)The impact of funding on the 5G innovation ecosystemScientometrics10.1007/s11192-024-04954-z129:4(2037-2056)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s11192-024-04954-z
Show More Cited By

Recommendations

The funding factor: a cross-disciplinary examination of the association between research funding and citation impact

This paper intends to illuminate the relationship between science funding and citation impact in seven STEMM disciplines (science, technology, engineering, mathematics, and medicine). Using a regression model with Heckman bias correction, we find that ...
Funding acknowledgement analysis: an enhanced tool to investigate research sponsorship impacts: the case of nanotechnology

There is increasing interest in assessing how sponsored research funding influences the development and trajectory of science and technology. Traditionally, linkages between research funding and subsequent results are hard to track, often requiring ...
How to boost scientific production? A statistical analysis of research funding and other influencing factors

This paper analyzes the impact of several influencing factors on scientific production of researchers. Time related statistical models for the period of 1996 to 2010 are estimated to assess the impact of research funding and other determinant factors on ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Journal of the Association for Information Science and Technology

Journal of the Association for Information Science and Technology Volume 67, Issue 3

March 2016

263 pages

ISSN:2330-1635

EISSN:2330-1643

Issue’s Table of Contents

Publisher

John Wiley & Sons, Inc.

United States

Publication History

Published: 01 March 2016

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

El-Ouahi J(2024)Research funding in the Middle East and North Africa: analyses of acknowledgments in scientific publications indexed in the Web of Science (2008–2021)Scientometrics10.1007/s11192-024-04983-8129:6(2933-2968)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s11192-024-04983-8
Li MWang Y(2024)Influence of political tensions on scientific productivity, citation impact, and knowledge combinationsScientometrics10.1007/s11192-024-04973-w129:4(2337-2370)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s11192-024-04973-w
van der Pol J(2024)The impact of funding on the 5G innovation ecosystemScientometrics10.1007/s11192-024-04954-z129:4(2037-2056)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s11192-024-04954-z
Gök AKaraulova M(2024)How “international” is international research collaboration?Journal of the Association for Information Science and Technology10.1002/asi.2484275:2(97-114)Online publication date: 4-Jan-2024
https://dl.acm.org/doi/10.1002/asi.24842
Lou WHe JZhang LZhu ZZhu Y(2023)Support behind the scenes: the relationship between acknowledgement, coauthor, and citation in Nobel articlesScientometrics10.1007/s11192-023-04803-5128:10(5767-5790)Online publication date: 22-Aug-2023
https://dl.acm.org/doi/10.1007/s11192-023-04803-5
Mo ZYujie ZJiasu LXiaowen T(2022)Early firm engagement, government research funding, and the privatization of public knowledgeScientometrics10.1007/s11192-022-04448-w127:8(4797-4826)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1007/s11192-022-04448-w
Liang ZMao JLu KBa ZLi G(2021)Combining deep neural network and bibliometric indicator for emerging research topic predictionInformation Processing and Management: an International Journal10.1016/j.ipm.2021.10261158:5Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1016/j.ipm.2021.102611
Dong JLiu JLiu T(2021)The impact of top scientists on the community development of basic research directed by government funding: evidence from program 973 in ChinaScientometrics10.1007/s11192-021-04092-w126:10(8561-8579)Online publication date: 1-Oct-2021
https://dl.acm.org/doi/10.1007/s11192-021-04092-w
Roshani SBagherylooieh MMosleh MCoccia M(2021)What is the relationship between research funding and citation-based performance? A comparative analysis between critical disciplinesScientometrics10.1007/s11192-021-04077-9126:9(7859-7874)Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1007/s11192-021-04077-9
Tonta YAkbulut M(2020)Does monetary support increase citation impact of scholarly papers?Scientometrics10.1007/s11192-020-03688-y125:2(1617-1641)Online publication date: 1-Nov-2020
https://dl.acm.org/doi/10.1007/s11192-020-03688-y
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents