Abstract
Serious concerns have been raised on the potentially negative impact of public measures to contain the COVID-19 pandemic on academic research, including the closure of research facilities, and the challenges of lockdown. However, it is unclear whether COVID-related mobility restrictions have penalized academic productivity, and if this is the case, whether it has had an equal impact on all research areas and countries. Here, we examined about 9.2 million submissions to 2689 Elsevier journals in all research areas in 2018–2021 and estimated the impact of anti-COVID mobility restriction policies on submissions to journals. Results showed that anti-contagion public measures had a positive impact on academic productivity. However, submission patterns changed more in non-Western academic countries, with the exception of Italy, which had stringent lock-down measures. During the early stages of the pandemic, the abnormal peak of submission was dominated by health & medical researchers, whereas later, there was an increase in submissions to social science & economics journals. Although anti-contagion public measures have contributed to change academic work, it is difficult to estimate whether they will have any potentially long-term effect on the academic community- either positive or negative.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
The COVID-19 pandemic has caused an increasing demand for research with abnormal requests for preprints, journal submissions, and COVID-related publications (Fraser, 2021; Abramo et al., 2022; Watson, 2022). However, this so called “covidization of research” has not affect all academic groups equally (Ioannidis et al., 2022). For instance, research has shown that during the first wave of the pandemic, junior women submitted proportionally fewer manuscripts than men (Squazzoni, 2021; Madsen et al., 2022), and that this penalty was even more prominent for junior women working in less prestigious academic organisations located in less gender-equal countries (Kwon et al., 2023). Studies also found that academics doing COVID-related research had the easiest and fastest publications compared to those doing non-COVID related research (Aviv-Reuven & Rosenfeld, 2021), thus determining various potentially intertwined forms of inequality (Santos, 2022).
On the one hand, the pandemic had a negative impact on the work habit and routine of many organizations and institutions due to difficult access to lab facilities, suspension of fieldwork, and competing demands from family obligations due to home-schooling and parental care (Petts et al., 2021; EU Commission, 2023; Esquivel et al., 2023). On the other hand, the public measures to contain the pandemic, including especially strong mobility restrictions, could also have potentially benefitted certain academics by providing time to complete existing work or adapting existing research to new opportunities. As for employees in other sectors (Deole et al., 2023), working from home might have caused reduced working time for busy parents, but saving time for mobility and face-to-face meetings could have benefited those who could either rely on family support or did not have children or old parents to care for (Carr, 2021; Parlangeli et al., 2022).
However, it is difficult to assess these heterogeneous effects of the pandemic at the global level without a careful comparison of pre- and post-pandemic data that considers all research areas. Here, we have tried to fill this gap by examining the impact of the COVID-19 pandemic with full and complete data on about 9,2 million submissions to 2689 Elsevier journals from 2018 to 2021. We reconstructed academic origins from the affiliation of each author and assigned a research area to each submission via journal information.
This allowed us to reconstruct the growth rate of submissions and its global temporal patterns before and during the pandemic per country of authors’ affiliation and research area. Furthermore, we considered heterogeneity effects on submissions due to country-level anti-contagion measures. This permitted us to estimate the effect of the pandemic on academic productivity more systematically than in previous research.
Methods
Data
Our dataset included about 9.2 million submissions to 2689 Elsevier journals from January 1, 2018 to May 31, 2021 in four research areas: Health & medicine sciences (HMS), Life sciences (LS), Physical sciences (PS), and Social sciences & economics (SSE) (Table 1). Data access required a confidential agreement to be signed on 12th May 2020 between Elsevier and each author of this study (Squazzoni et al., 2017). We used the e-mail (sometimes various e-mails) associated with each submission’s author in the different submission systems used by Elsevier journals (i.e., Editorial Manager, Elsevier Editorial System, and EVISE) to reconstruct the country of affiliation of each author for each submission. Each submission was also assigned a unique research area depending on the journal which it was submitted to. To distinguish COVID related and non-related manuscripts, we used an internal Boolean flag from the manuscript submission systems used by journals in the Elsevier data. This allowed us to use a taxonomy of terms related to diseases caused by the same family of viruses to track back COVID-19 related manuscripts before the start of the pandemic (Squazzoni, 2021).
To estimate the submission growth rate over time, we performed a seasonal trend decomposition using Loess, isolated periodical events, and focused on unexplained events in our time series. This decomposition was performed by using the STL method from the StatsFootnote 1 library in R.Footnote 2 To calculate and compare changes between different periods of our time series, we used the Autocorrelation-based dissimilarity method from the package TSdist.Footnote 3
To control for mobility restrictions, we used the COVID-19 stringency index, i.e., a composite measure based on nine policy responses. These nine measures included: school closures, workplace closures, cancellation of public events, restrictions on public gatherings,; closures of public transport, stay-at-home requirements, public information campaigns, restrictions on internal movements, and international travel controls. Calculated on a daily basis, the index considered the mean score of all nine metrics and takes a value between 0 and 100. A higher score indicates the introduction of stricter policy responses to the pandemic (Hale, 2021).
Results
The COVID-19 pandemic caused an abnormal rate of manuscript submissions to journals compared to the pre-pandemic period, mostly concentrated between February and May 2020 (see Fig. 4, Table 1 in the Methods Section, and Table 3 in the Appendix). This abnormal trend showed significant country- and research area-specific differences. While before the pandemic, the number of submissions from China, India, the United States, and Western European countries was comparable, the first wave of the COVID-19 pandemic created a more prominent growth rate of submissions from authors from China and India (+ 55% of submissions in 2018–2020: 674,180 submissions in 2018 vs. 1,048,717 in 2020) compared to authors from the United States and Europe (+35% of submissions in 2018–2020: 406,336 submissions in 2018 vs. 542,706 in 2020). We also found higher growth rates from certain peripheral countries, such as Bangladesh, Vietnam, Saudi Arabia, Indonesia, and Turkey (see the left panel in Figs. 1 and 4 in the Appendix).
To provide a more robust measurement of the submission patterns that also considered seasonality and temporal trends, we first calculated an autocorrelation-based dissimilarity index between pre and post-pandemic submission time series per country. Given that this measurement considers seasonality of submission trends, it could be considered as a proxy of how academics restructured their own research agenda and manuscript submission routines. Our results suggest that the countries most positively affected by the pandemic, were Nigeria (2.22), Bangladesh (1.95), Argentina (1.88), and Italy (1.87) (see the right panel in Fig. 1).
Furthermore, besides regional effects, this abnormal trend did not have the same impact in all research areas. Indeed, in the earlier onset of the pandemic, i.e., in February–May 2020, in correspondence with the highest peaks of submission growth rate, submissions were also mostly addressed to journals in Health & Medical Sciences. However, in later periods, research directed to Social Sciences & Economics journals became more prominent, probably reflecting the persistent interest of the academic community in examining the psychological, socioeconomic, and political implications of the pandemic (see Fig. 2).
To check the hypothesis that the anti-COVID mobility restriction measures introduced, i.e., mobility restrictions, could have affected these patterns, we estimated a mixed-effect model where we used the stringency index calculated by the Oxford COVID-19 Government Response Tracker (Hale et al., 2020). This was to predict the auto-correction distance between pre-post pandemic submission trends.
Table 2 shows a positive and significant effect of the stringency index on the abnormal rates of submissions to journals, even when controlling for other variables, including the growth rate trends, and country-specific factors. For each increase of 10 in the value of the stringency index, we found an increase of 1% in the auto-correlation distance between pre- and post-pandemic submission trends. This suggests that mobility restrictions and other anti-contagion public measures at the country level had an effect on disrupting submission trends.
Figure 3 shows a visual representation that confirms that countries with the strongest mobility restriction policies were also those where the country-specific pattern of submissions to journals varied more compared to their own pre-pandemic trends (e.g., Italy, Argentina, and India). In Northern European countries, where mobility restrictions were either not contemplated or only softly recommended, the effect of the pandemic on changing pre and post-pandemic submission patterns was weaker (e.g., Finland, Sweden, and Denmark) (more country-based detail in Table 6 in the Appendix). Although there is heterogeneity in the effect for certain specific countries, results suggest that generally strongest mobility restriction policies had a positive effect on submission trends.
Discussion and conclusions
Previous evidence has suggested that the annual growth rate of scientific publications globally was around 4% percent (Bornmann et al., 2021). Our results showed that the COVID-19 pandemic and the associated mobility restrictions boosted this trend with huge submission growth rates for certain countries. Besides the increasing demand for research to tackle the multi-faceted challenges of the global pandemic, previous research has suggested the anti-contagion measures have contributed to shape new routines and changing practices, including lecturing from home, and reduced time from daily work commuting and in-person meetings, which could have boosted productivity in the short-term (Pellegrini et al., 2020; Commission, 2023).
Our findings showed that mobility and anti-contagion restrictions had the positive effect of boosting manuscript submissions to journals and disrupting previous submission trends and their seasonality. We also found prominent country and research area-specific effects, with the emergence of a relatively new geography of the pandemic research with certain peripheral countries and non-Western regions in a prominent position. In general, with the exception of Italy at the frontline of COVID-19 spread, and therefore introducing stringent anti-contagion policies, submissions grew especially from non-Western countries (Santos, 2022).
Furthermore, we found an interesting area-specific trend in COVID-related research. While the highest peaks of the submission growth rate were initially addressed to journals in Health & Medical Sciences, a later growth of submissions was also visible for submissions to Social Sciences & Economics journals. This showed the changing focus of research from health issues to more general societal implications of the pandemics, echoing previous studies on temporal trend dynamics of adaptation of the scientific community to the complex nature of the pandemic crisis (Santos., 2022).
This said, our study has also certain limitations. The assumption of our study is that the country of affiliation of authors and their country of residence during the pandemic in 2020/2021 were the same. It is possible that some global academics could have decided to relocate in their home country to reunite with their family, thus anticipating the possible effect of restrictions either in the country of their affiliation or in the country of their family. However, given the large scale nature of our dataset, we believe that these cases would be randomly distributed across countries in the sample without affecting our results.
Furthermore, our study cannot help to estimate the long-term effect of these changes in the submission patterns. Although previous survey research has emphasized the expected long-lasting effects of the pandemic on the academic community (Gao et al., 2021), understanding the long-term implications of these new practices, including the co-existence of on/off line meetings and lectures, the new work-life balance of academics, and the expected reduction of international conferences, in terms of potential changes in the annual growth of scientific activities and their unequal distribution between research areas, requires more research (Ashencaen et al., 2021; Jack & Glover, 2021).
Furthermore, by providing new opportunities for fast publications, the abnormal submission growth due to the pandemic could have compromised ethical standards of research integrity, and weakened the filters of peer review, thus undermining the quality of scientific publications (Bauchner et al., 2020; Horbach, 2021; Faust et al., 2023). Research that considers the interplay of these multiple, quantitative and qualitative factors is also needed to inform possible interventions and understand how to maintain rigorous standards of research and scholarly communication in periods of global shocks.
Data availability
The dataset for replication is available at this link: https://doi.org/10.7910/DVN/DUBP4U.
References
Abramo, G., D’Angelo, C. A., & Di Costa, F. (2022). How the COVID-19 crisis shaped research collaboration behaviour. Scientometrics, 127(3), 5053–5071.
Ashencaen Crabtree, S., Esteves, L., & Hemingway, A. (2021). A “new (ab) normal”?: Scrutinising the work-life balance of academics under lockdown. Journal of Further and Higher Education, 45(9), 1177–1191.
Aviv-Reuven, S., & Rosenfeld, A. (2021). Publication patterns’ changes due to the COVID-19 pandemic: a longitudinal and short-term scientometric analysis. Scientometrics, 126, 6761–6784. https://doi.org/10.1007/s11192-021-04059-x
Bauchner, H., Fontanarosa, P. B., & Golub, R. M. (2020). Editorial evaluation and peer review during a pandemic: How journals maintain standards. JAMA, 324(5), 453–454. https://doi.org/10.1001/jama.2020.11764
Bornmann, L., Haunschild, R., & Mutz, R. (2021). Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases. Humanities and Social Sciences Communications, 8(1), 224. https://doi.org/10.1057/s41599-021-00903-w
Carr, R. M., et al. (2021). Academic careers and the COVID-19 pandemic: Reversing the tide. Science Translational Medicine, 13(584), eabe7189. https://doi.org/10.1126/scitranslmed.abe7189
EU Commission Directorate-General for Research and Innovation. (2023). The COVID-19 impact on gender equality in research & innovation (Publications Office of the European Union)
Deole, S. S., Deter, M., & Huang, Y. (2023). Home sweet home: Working from home and employee performance during the COVID-19 pandemic in the UK. Labour Economics, 80, 102295. https://doi.org/10.1016/j.labeco.2022.102295
Esquivel, A., Marincean, S., & Benore, M. (2023). The effect of the COVID-19 pandemic on stem faculty: Productivity and work-life balance. PLOS ONE, 18(1), 1–10. https://doi.org/10.1371/journal.pone.0280581
Faust, A., Saksone, L., Weigold, S., Woydack, L., & Strech, D. (2023). Pandemics and research ethics. An overview of central challenges (pp. 149–163). Heidelberg: Springer. https://doi.org/10.1007/978-3-662-66872-6-11
Fraser, N., et al. (2021). The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape. PLOS Biology, 19(4), 1–28. https://doi.org/10.1371/journal.pbio.3000959
Gao, J., Yin, Y., Myers, K. R., Lakhani, K. R., & Wang, D. (2021). Potentially long-lasting effects of the pandemic on scientists. Nature Communications. https://doi.org/10.1038/s41467-021-26428-z
Hale, T., Webster, S., Petherick, A., Phillips, T. & Kira, B. (2020) Oxford COVID-19 government response tracker https://covidtracker.bsg.ox.ac.uk/. University of Oxford, Blavatnik School of Government.
Hale, T., et al. (2021). A global panel database of pandemic policies (Oxford COVID-19 government response tracker). Nature Human Behaviour, 5(4), 529–538. https://doi.org/10.1038/s41562-021-01079-8
Horbach, S. P. (2021). No time for that now! Qualitative changes in manuscript peer review during the COVID-19 pandemic. Research Evaluation, 30(3), 231–239.
Ioannidis, J. P. A., Bendavid, E., Salholz-Hillel, M., Boyack, K. W., & Baas, J. (2022). Massive covidization of research citations and the citation elite. Proceedings of the National Academy of Sciences, 119(28), e2204074119.
Jack, T., & Glover, A. (2021). Online conferencing in the midst of COVID-19: An “already existing experiment” in academic internationalization without air travel. Sustainability: Science, Practice and Policy, 17(1), 292–304.
Kwon, E., Yun, J., & Han Kang, J. (2023). The effect of the COVID-19 pandemic on gendered research productivity and its correlates. Journal of Informetrics, 17(1), 101380. https://doi.org/10.1016/j.joi.2023.101380
Madsen, E. B., Nielsen, M. W., Bjørnholm, J., Jagsi, R., & Andersen, J. P. (2022). Meta-research: Author-level data confirm the widening gender gap in publishing rates during COVID-19. eLife, 11, e76559. https://doi.org/10.7554/eLife.76559
Montero, P. & Vilar, J.A. (2014). Tsclust: An R package for time series clustering. Journal of Statistical Software. 62(1), 1–43 https://doi.org/10.18637/jss.v062.i01
Parlangeli, O., Palmitesta, P., Bracci, M., et al. (2022). University teachers during the first lockdown due to SARS-COV-2 in Italy: Stress, issues and perceptions of misconduct. Science Engineering Ethics, 28(9), 292–304.
Pellegrini, M., Uskov, V., & Casalino, N. (2020). Reimagining and re-designing the post-COVID-19 higher education organizations to address new challenges and responses for safe and effective teaching activities. Law and Economics Yearly, Review Journal-LEYR Queen Mary University London, UK, 9(part1), 219–248.
Petts, R. J., Carlson, D. L., & Pepin, J. R. (2021). A gendered pandemic: Childcare, homeschooling, and parents’ employment during COVID-19. Gender Work Organization, 28(S2), 515–534. https://doi.org/10.1111/gwao.12614
Santos, S. B., et al. (2022). Discovering temporal scientometric knowledge in COVID-19 scholarly production. Scientometrics, 127(3), 1609–1642.
Squazzoni, F., et al. (2021). Gender gap in journal submissions and peer review during the first wave of the COVID-19 pandemic. A study on 2329 Elsevier journals. PLOS ONE, 16(10), 1–17. https://doi.org/10.1371/journal.pone.0257919
Squazzoni, F., Grimaldo, F., & Marušić, A. (2017). Publishing: Journals could share peer-review data. Nature. https://doi.org/10.1038/546352a
Watson, C. (2022). Rise of the preprint: How rapid data sharing during COVID-19 has changed science forever. Nature Medecine, 28, 2–5. https://doi.org/10.1038/s41591-021-01654-6
Acknowledgements
Bahar Mehmani would like to thank Naomi Lee, senior executive editor of the Lancet for insightful discussion on the nature of the submission trend. We also would like to thank Marina Martin-Gisbert for her preliminary work on describing the temporary submissions series and, concretely, for her help on data visualization, creating the plots shown in Figs. 4 and 5.
Funding
Open access funding provided by Università degli Studi di Milano within the CRUI-CARE Agreement. Daniel García-Costa and Francisco Grimaldo are supported by the Spanish Ministry of Science, Innovation and Universities (MCIU), the Spanish State Research Agency (AEI), and the European Regional Development Fund (ERDF) under project RTI2018-095820-B-I00. Flaminio Squazzoni is supported by a PRIN-MUR (Progetti di Rilevante Interesse Nazionale - Italian Ministry of University and Research) grant (Grant Number: 20178TRM3F001). Funders had no role in the study design.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Bahar Mehmani declares a competing interest, being currently employed as Reviewer Experience Lead at Elsevier. However, she had no access to the database, elaborated any version of the dataset, or was involved in data analysis. The authors declare no other competing interests.
Appendix
Appendix
Table 5 shows the number of COVID-19 vs. non-COVID-19 submissions by year and research areas. Note that, following Squazzoni et al. (2021), we used data from the manuscript submission systems used by Elsevier journals to distinguish COVID related and non-related manuscripts. This allowed us to track any manuscript focusing on diseases caused by a COVID-like family of viruses before the start of the pandemic. Results suggest a more prominent increase of COVID-related submissions in HMS journals in 2020 compared to 2019 and in SSE journals in 2021 compared to 2020 and 2019, as mentioned in the main text.
The decomposition of the series based on a Holt-Winter’s model indicated a growing trend of submissions in the whole period, with an acceleration in 2020 (Fig. 4). Unsurprisingly, the trend showed a seasonality dynamics, with a sharp reduction in submissions from the Northern hemisphere during the winter break. Despite the overall growing trend, during the first wave of the COVID-19 pandemic, i.e., Feb–May 2020, we found important positive residuals. This suggests a breakthrough in the long-term trends of journal submissions: The first wave of the COVID-19 pandemic caused a system-level shock, which was eventually absorbed by the system by the end of 2020, where the number of submissions returned to pre-pandemic trends. To better analyse this, we compared submissions in 2019, 2020, and the first five months of 2021 with those from 2018, which were used as baseline (note that we only used the first five months of 2018 for comparison to 2021). We then built a measure of dissimilarity based on the auto-correlation distance function that reflected trend differences and variations for each period.
Note that the autocorrelation-based dissimilarity is a measure of the difference between two time series, and is calculated as the dissimilarity between their estimated autocorrelation coefficients (Montero et al., 2014). This index takes an unbounded value between 0 and an infinite number, with higher values indicating higher dissimilarities between the pair of time series. For our time series, the maximum values of the autocorrelation-based dissimilarity were around 2.5.
Figure 5 shows variations in time series for the first 50 world countries for the highest numbers of submissions during the pandemic. The top panel, i.e., 2019–2018 comparison, shows the pre-pandemic increase trend in the number of submitted manuscripts. The middle panel (2020 vs 2018) shows a significant departure from the previous trend. Not only did the number of submissions accelerate, there were abnormal positive values in most countries severely affected by the COVID-19 pandemic. Finally, the 2021–2018 comparison (bottom panel) shows a return to pre-pandemic submission trends for most countries.
Table 4 shows that the growth rate of submissions and the auto-correlation distance were both more prominent in Africa, South America, and Asia. In Europe, the auto-correlation distance was mostly due to Southern Europe, with countries such as Italy and Spain with the most prominent distance from previous trends (see also Table 5 for a list of countries by growth rate and Table 6 for a list of countries by auto-correlation distance).
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
García-Costa, D., Grimaldo, F., Bravo, G. et al. The silver lining of COVID-19 restrictions: research output of academics under lockdown. Scientometrics 129, 1771–1786 (2024). https://doi.org/10.1007/s11192-024-04929-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-024-04929-0