Open AccessReview

Meta-Research in Biomedical Investigation: Gaps and Opportunities Based on Meta-Research Publications and Global Indicators in Health, Science, and Human Development

Ivan David Lozada-Martinez

^1,2,*

David A. Hernandez-Paz

^2,3

Ornella Fiorillo-Moreno

^4,5

Yelson Alejandro Picón-Jaimes

⁶

and

Valmore Bermúdez

^7,*

Biomedical Scientometrics and Evidence-Based Research Unit, Department of Health Sciences, Universidad de la Costa, Barranquilla 080002, Colombia

Center for Meta-Research and Scientometrics in Biomedical Sciences, Barranquilla 080002, Colombia

Grupo Prometheus y Biomedicina Aplicada a las Ciencias Clinicas, School of Medicine, Universidad de Cartagena, Cartagena 130001, Colombia

⁴

Clínica Iberoamérica, Barranquilla 080001, Colombia

⁵

Clínica El Carmen, Barranquilla 080001, Colombia

⁶

Facultat de Ciències de la Salut Blanquerna, Universitat Ramon Llull, 08025 Barcelona, Spain

⁷

Centro de Investigaciones en Ciencias de la Vida, Facultad de Ciencias de la Salud, Universidad Simón Bolívar, Barranquilla 080001, Colombia

Authors to whom correspondence should be addressed.

Publications 2025, 13(1), 7; https://doi.org/10.3390/publications13010007

Submission received: 17 November 2024 / Revised: 3 January 2025 / Accepted: 8 February 2025 / Published: 10 February 2025

Download

Browse Figures

Figure 1
Global distribution of meta-research-related publications in biomedicine by country. Each country is shaded according to the total number of publications, with darker shades representing a higher volume. The United States leads in meta-research output, as indicated by the darkest red shading, with 2447 publications. Other countries with high publication volumes, shown in varying shades of blue, include the United Kingdom, Canada, and several European and Asian countries. In contrast, countries shaded in light grey lack reported data in this dataset. "> Figure 2
Global and categorical analysis of meta-research-related publications in biomedicine. (a) World distribution of meta-research publications by country from 1948 to 2024. (b) Total citations by country for meta-research publications indicate the impact of each country’s contributions over the same period. (c) Distribution of publication types within meta-research, categorised into original research articles, reviews, editorials, and other document types, illustrating the predominant publication formats for the top five countries with the highest publication volumes. (d) Journals with the highest number of meta-research publications from 1970 to 2024. (e) Publishing groups with the highest number of meta-research publications from 1981 to 2024. "> Figure 3
Visualisation of the regression model between MP and global health, science, and human development indicators. (a) Association between the logarithm of meta-research publications count and the summed HDI, with fitted regression line and confidence band. (b) Scatter plot with regression between the meta-research articles’ logarithm and completed clinical trials’ logarithm. (c) Plot of the relationship between the logarithm of meta-research articles (with a one-year lag) and the logarithm of completed clinical trials. HDI: Human Development Index. MP: Meta-Research-related Publications in Biomedicine. ">

Versions Notes

Abstract

Meta-research in biomedical science is crucial for ensuring rigour, relevance, and transparency in an era marked by the exponential growth of scientific publications. This study examines global and historical trends in meta-research activities within biomedicine and investigates their relationship with health, science, and human development indicators. A systematic analysis of 9633 publications from Scopus, Web of Science, and PubMed was conducted, focusing on publication volume, citation impact, and geographic distribution. Regression analyses reveal a significant positive association between meta-research activity and the Human Development Index (HDI), suggesting that meta-research contributes to societal advancement by enhancing evidence-based decision-making in health. However, no association was found between meta-research output and research and development (R&D) expenditure, reflecting the minimal resource requirements of secondary data-driven studies compared to primary or experimental research. Meta-research activity correlates positively with clinical trial completion, indicating its role in refining study designs and addressing evidence gaps. These findings highlight the importance of expanding meta-research in underrepresented regions to promote equity in scientific advancement and improve the reliability of biomedical knowledge. This result underscores the need for targeted support for meta-research, particularly in low- and middle-income countries with limited scientific infrastructure and resources for new knowledge generation.

Keywords:

publishing; biomedical research; meta-research; evidence gaps; sustainable development

1. Introduction

Despite significant variation across fields, the rate of scientific publications has steadily increased over the past century, making it progressively more challenging to track the evolution of broad areas such as biomedical literature (Larsen & von Ins, 2010; Bornmann & Mutz, 2015). The PubMed database currently houses over 21 million English-language biomedical and life sciences articles and has archived more than 35 million scientific papers from the last 50 years (González-Márquez et al., 2024). This fact positions biomedical research as the most prolific area of research globally. Non-linear dimensionality reduction methods such as t-distributed Stochastic Neighbour Embedding (t-SNE), coupled with transformer-based large language models, have been employed to analyse this vast body of data to gain a broad perspective on the evolution of biomedical literature (González-Márquez et al., 2024). However, while these methods are useful for identifying interesting patterns, they do not comprehensively address aspects related to scientific rigour, methodological quality, relevance, or impact. Concern for the validity of scientific evidence in medical and health sciences has been a topic of discussion for several years, which has led to the proposal of novel but little-explored mixed research methods that evaluate various dimensions and determinants of the scientific method applied to health research (Ioannidis, 2005).

In this context, meta-research has emerged as a promising field to accelerate scientific progress by researching research itself. This discipline focuses on five key areas: how to conduct, communicate, verify, evaluate, and reward scientific research (Ioannidis et al., 2015). Meta-research, alongside methodologies like bibliometrics (a branch of scientometrics) (Hicks et al., 2015) and evidence-based research (EBR), which aims to promote a more systematic, transparent, and efficient way of generating high-quality and relevant results (Robinson et al., 2021), are essential to ensuring scientific rigour in an era of massive, often low-quality, and non-reproducible research.

Despite the importance of understanding this concept, it has been demonstrated that few research groups or individual researchers implement them (Ioannidis, 2005, 2008, 2010, 2014). This oversight represents a significant gap, given the vast array of areas and disciplines that comprise medical and health sciences, as well as the high level of heterogeneity and responsibility required to apply the highest quality evidence for evidence-based decision-making in healthcare. However, although meta-research evaluates research itself, its behaviour in biomedical sciences has been scarcely examined since its inception and evolution. Therefore, this study aims to (1) quantify and characterise the global and historical meta-research activity within biomedical research and (2) identify the impact of meta-research publications on global indicators in health, science and human development.

2. The Role and Importance of Meta-Research in Biomedical Sciences

Science has become a global industry, with over 1.5 million scientific articles published only on biomedicine and life sciences yearly (González-Márquez et al., 2024); however, the certainty and applicability of major discoveries are still a concern (Ioannidis, 2014). It has been said that “for most study designs and settings, it is more likely for a research claim to be false than true”, largely due to factors such as study and effect sizes, the number of hypotheses tested, flexibility in study design and outcome definitions, conflicts of interest, and competition in popular research fields (Ioannidis, 2005). But, even if a research discovery is true, the observed effect size would probably be inflated due to underpowered studies and statistical significance thresholds, flexible analysis and selective reporting, or interpretation bias (Ioannidis, 2008).

Over time, even studies considered to be at the top of the evidence pyramid have demonstrated inconsistencies. A clear example is the use of corticosteroids for acute bacterial meningitis (Ioannidis, 2010). In 1994, it was reported that there was “no question about benefits, but beware of harms” (Ioannidis et al., 1994). By 1997, the recommendation shifted to “definite benefit only for some bacteria, then limit to 2 days to avoid harm” (McIntyre, 1997). Then, in 2003, there was a “definite benefit only for children, no increase in harm”, followed by a correction later that year indicating that “benefit is also seen in adults” (Ioannidis, 2010). In 2007, findings suggested: “benefit in high-income countries, but not in low-income countries” (Beek et al., 2008). By 2009: “clear benefit, give it to all” (Vardakas et al., 2009). However, in 2010: “no benefit at all”. The latest systematic review in the Cochrane Library in 2015 stated: “Corticosteroids significantly reduced hearing loss and neurological sequelae, but did not reduce overall mortality. Data support the use of corticosteroids in patients with bacterial meningitis in high-income countries. We found no beneficial effect in low-income countries” (Brouwer et al., 2015).

In 2013, Fanelli et al. (2017) collected 3042 meta-analyses from the Web of Science database, covering the 22 disciplinary categories used by Thompson Reuters’ Essential Science Indicators database. They assessed various biases and the underlying sociological and psychological factors. Their findings indicate that small-study effects, grey literature bias, and citation bias are the most prevalent sources of bias. Additionally, they observed a negative association between authors’ career length and effect size, with early-career researchers being more likely to overestimate effects (Fanelli et al., 2017). Meta-research has enabled a critical appraisal of scientific findings, shedding light on the true nature of data and fostering more informed, evidence-based decision-making.

Despite its significance, the correlation between global biomedical meta-research activity and global indicators in health and science has not been previously studied. Such an analysis can reveal structural patterns in scientific production, highlighting inequalities in the distribution of knowledge and access to research resources. Consequently, this study provides a conceptual framework to guide the development of more inclusive policies and practices. In doing so, it contributes to bridging the gap in understanding the existing methodological and conceptual limitations related to the study of meta-research in health.

3. Global Trends in Meta-Research-Related Publications in Biomedicine

Using a specified search strategy, we searched the Scopus, Web of Science and PubMed databases for all meta-research-related publications in biomedicine (MP) and its synonyms (Supplementary Material). After a manual review of inclusion and exclusion criteria (specified in the supplementary material), the resulting dataset included 9633 articles with information on the journal, country, publication year, total citations, publishing group, and document type. We then filtered for articles with a reported first author’s country, resulting in 6505 MP spanning from 1946 to 2024 across a total of 88 countries. The total number of MP was plotted on a world map (Figure 1) and as a bar plot stratified by years (Figure 2a). The top three countries with the highest volume of MP—the USA, UK, and Canada—accounted for 56.37% (n = 3667) of the total MP, with the USA alone contributing 37.61% (n = 2447). This geographic distribution highlights the concentration of meta-research efforts primarily in North America and Western Europe.

We next analysed citation counts by country, identifying the 15 most-cited countries (Figure 2b). The USA and UK comprised 54.18% (n = 66,318) of total citations across all years. Notably, while the Netherlands ranked seventh in publication volume (n = 171), it was third in citation count (n = 13,253; 10.83%). China, the fourth-largest MP producer (n = 321), ranked sixth in citation counts (n = 5104; 4.17%). Despite high publication counts, countries like South Korea, India, and Brazil did not appear among the top 15 most-cited countries. Conversely, Denmark, Sweden, and Mexico were among the top 15 by citation count but not by publication volume.

To further explore MP characteristics, we analysed document types across all countries in the dataset, identifying the following categories: Articles (n = 3624), Reviews (n = 1198), Editorials (n = 808), Letters (n = 435), Notes (n = 378), and Short Surveys (n = 62). We examined the distribution of these categories among the top five countries by MP volume (Figure 2c). The USA and UK had the largest proportions across all document types. However, when normalising for the total document types within each country, China exhibited a higher proportion of Articles (68.53%) and Reviews (24.92%) compared to other countries (Articles: 52–57%; Reviews: 15–19%). China also had the lowest proportion of Editorials (1.55%), with other countries ranging between 8–15%.

We further assessed MP trends by year across journals, including entries lacking country data, resulting in 9631 articles across 3119 journals (unfiltered by individual journals). The top five journals, each classified as Q1 by the Scimago Journal and Country Rank in 2023, accounted for 11.51% of MP publications. European Food Safety Authority (EFSA) Journal had the highest MP count (n = 644; 6.68%), with all top five journals based in the UK except for the Journal of Clinical Epidemiology, based in the USA (Figure 2d). Filtering the dataset by publishing group (PG) yielded 5623 articles across 559 PGs. Elsevier accounted for 12.8% (n = 720) of the MP, while the top five PGs collectively published 36.74% (n = 2066) of the MP (Figure 2e). Additional information regarding these descriptions can be found in the supplementary material (Tables S1–S3).

4. Comparison Between Meta-Research Activity and Indicators in Global Health, Science and Human Development

Regression models were conducted in R software (version 4.4.0) using MP and specific global indicators of health, science, human development, and innovation to examine the relationship between them. The R code used for the analyses is available at [https://doi.org/10.5281/zenodo.14592259]. The data required to reproduce the analyses can be provided upon reasonable request. First, a comparison was made between meta-research activity and the Human Development Index (HDI). We analysed data covering the period from 1990 to 2022. HDI data was sourced from the United Nations Development Programme (https://tinyurl.com/ycxh9mnn) (accessed on 9 November 2024), representing a composite measure of longevity, education, and standard of living. Only indicators from 85 of the 88 countries with MP were included in the regression model, using the annual total HDI for each country. The annual total of meta-research publications was calculated based on the sources described previously, and the same time period was used for subsequent analyses.

Given the growth patterns in the dependent variable (HDI) and the independent variable (number of publications), a logarithmic transformation was applied to the article count to reduce skewness and enhance model interpretability. We then fitted a linear regression model with the log-transformed publications count as the predictor and HDI as the outcome variable. This approach was chosen over more complex time-series analyses or first-difference models, as our primary goal was to capture the general association between the variables rather than focusing on annual fluctuations.

The logarithmic regression model revealed a positive and significant coefficient for meta-research publications count (β = 6.5454, p < 0.001), indicating a positive association between increased meta-research activity and HDI (Figure 3a). The model’s adjusted r² was 0.7188, suggesting that approximately 71.88% of the variability in HDI can be explained by meta-research activity. Residual analysis showed that normality assumptions were not fully met (p = 0.0039 in the Shapiro-Wilk test), though no significant heteroscedasticity was detected (p = 0.091 in the Breusch-Pagan test). These results indicate that the model is reasonably robust; however, results should be interpreted cautiously due to the non-normality of the residuals.

A first-difference regression analysis between annual changes in publications count and HDI yielded no statistically significant results (p = 0.216) and thus was not included in the main analysis. This finding suggests that the association observed in the logarithmic model likely reflects an overall relationship across the study period rather than specific annual changes. As shown in Table 1, the regression results highlight a strong positive association between the number of meta-research publications and HDI.

To examine the relationship between meta-research activity and R&D expenditure (% of gross domestic product [GDP]) (https://tinyurl.com/ykdeecfy) (accessed on 9 November 2024), we analysed annual data from 84 of the 88 countries reporting MP spanning from 1996 to 2022. We employed correlation, linear regression, cointegration, Granger causality, and lagged regression analyses. Results showed no significant association between the number of meta-research publications and R&D expenditure, with a low Pearson correlation (r = 0.097) and non-significant regression coefficient (β = 0.0531, p = 0.631). Cointegration analysis found no long-term relationship, while the Granger causality test suggested a predictive link (p = 0.024). However, the lagged regression indicated no significant effect of meta-research activity on R&D expenditure in the following year (p = 0.845). In summary, no strong association was found between meta-research activity and R&D expenditure, though there is minor evidence of a predictive relationship. These findings suggest limited interaction between these indicators.

Data on completed clinical trials was obtained from ClinicalTrials.gov, encompassing 280,944 completed trials from multiple countries. Meta-research publications counts were obtained as previously described. This analysis assessed the association between annual meta-research activity and the annual total of completed clinical trials from 2005 to 2024. We first computed the Pearson correlation between the log-transformed counts of meta-research articles and completed clinical trials, followed by a linear regression analysis. Both variables were log-transformed to address skewness. Figure 3b illustrates the positive linear association, with a high correlation and significance (p < 0.001), suggesting that increased meta-research activity is associated with higher clinical trial activity rates.

Residual analysis indicated slight non-normality in the residuals (p < 0.05) but confirmed homoscedasticity (p = 0.376), suggesting that the model assumptions hold reasonably well. A Johansen cointegration test revealed the presence of a long-term equilibrium relationship between meta-research articles and completed trials, indicating that these variables move together over time. We conducted a lagged regression analysis to explore potential delayed effects using the prior year’s meta-research activity as a predictor of the current year’s clinical trials. This model showed a significant association, suggesting a one-year lagged effect of meta-research activity on trial completions (Figure 3c). Table 1 summarises the regression results, including the main coefficients, standard errors, and significance levels for both the linear and lagged models. Together, these analyses suggest a robust relationship between meta-research activity and clinical trial activity over the study period.

These findings suggest that meta-scientific activity conducted over time, aimed at promoting rigour, relevance, and appropriateness in generating new knowledge—particularly in health—shows a positive and significant relationship with human development and the execution of clinical trials. This association may be explained by identifying knowledge gaps in health and the timely proposition of solutions through the scientific method. Notably, there is no association between meta-scientific activity and research and development expenditure. This behaviour is expected, as meta-scientific work typically relies on secondary data, which requires fewer resources than primary or experimental data research, even when conducted on a large scale. These results could serve as evidence supporting the development of research lines and workgroups in low- and middle-income countries with low indicators of human and scientific development.

5. Gaps, Needs and Opportunities in Biomedical Meta-Research

The primary goal of science is to expand our knowledge base, ideally leading to practical applications that underscore its value. What began as the pursuit of a few dedicated dilettantes has evolved into a global profession, engaging scientists from diverse fields and nations. Today, science practice requires continuous optimisation to accelerate progress—an endeavour central to meta-research. One key area of meta-research involves examining how research is rewarded, with a particular focus on resource distribution—a critical yet understudied issue. Ironically, although research is fundamentally evidence-based (Faggion, 2021; Lund et al., 2020a, 2020b), allocating research funding, often involving billions of dollars, usually lacks an evidence-driven framework. This oversight leaves us uncertain about the most effective ways to allocate resources, which is a significant gap for several solutions that have been proposed (Ioannidis, 2011).

Meta-research approaches have not been uniformly implemented across all fields of knowledge, leaving notable gaps in areas such as medical ethics. For instance, Faggion (2021) analysed MP from 16 high-impact medical ethics journals and found fewer than 2% (n = 45) articles addressed meta-research topics, with key meta-research areas like reproducibility and incentives particularly underrepresented. One such MP (Wu et al., 2019) identified critical ethical reporting gaps, showing that only half of the prospective clinical studies reviewed had documented informed consent from participants. In methodological rigour, a study published in Developing World Bioethics (Rocha & Grisolia, 2018) found that 84% of toxicology studies from public laboratories reported positive associations between pesticides and cellular changes, while 79% of studies showing no such associations originated from private laboratories. These findings underscore the need for more extensive meta-research across scientific disciplines to enhance transparency and scientific integrity.

The results of this study highlight a significant correlation between meta-research activity in biomedicine and global indicators such as the HDI and the volume of clinical trials. These findings could underscore the critical role that meta-research plays in strengthening scientific rigour, improving methodological standards, and facilitating evidence-based decision-making in health sciences. The positive association with HDI (β = 6.5454, p < 0.001) may reflect how countries with higher meta-research publications have better human development. This outcome could support the hypothesis that nations prioritizing scientific progress have better progress in the areas of education, health, and quality of life.

The strong relationship between meta-research activity and clinical trials (β = 1.6056, p < 0.001) further reinforces the utility of meta-research in promoting clinical research and accelerating the implementation of clinical innovations. The observed one-year lagged effect (β = 1.4194, p < 0.001) suggests that meta-research not only informs but also stimulates subsequent clinical trials, enabling a feedback loop that fosters sustained scientific advancement. This interaction highlights the practical applications of meta-research as a tool to identify knowledge gaps and improve study designs, ultimately driving research efficiency and relevance.

Given these gaps, education on the fundamentals of conducting research on research itself from the early stages of a scientific career is not only an opportunity but a necessity. This foundation is essential to curtail the growing influx of low-quality scientific publications (Fabiano et al., 2024). Aligned with meta-research principles, the appraisal of new research should rely on comprehensive criteria, free from biases associated with traditional indicators like the journal impact factor. Science should be judged by its content, a principle that is especially crucial for shaping the values of future generations of scientists (Hicks et al., 2015; Zhang et al., 2017; Boury et al., 2023).

This study revealed intriguing associations regarding the interest in developing knowledge and hypotheses focused on rigour and relevance in research and its potential impact on scientific, human, and health development indicators. These findings suggest that strengthening standards in research, science, and health innovation could positively influence human and scientific development in the medium- to long-term (Djokoto, 2022; Qin et al., 2023). Therefore, designing new mission-oriented research policies and setting research priorities in health should consider incorporating a section on meta-science applied to biomedical research. This approach aims to improve the quality of future evidence, particularly in contexts with limited resources for research and development funding (Lozada-Martinez et al., 2022, 2023, 2024a, 2024b).

6. Conclusions

This study comprehensively analyses meta-research trends in biomedical science, underscoring the discipline’s role in fostering evidence quality and integrity globally. Key findings demonstrate a robust association between meta-research activity and human development, as evidenced by a significant relationship with the Human Development Index. This result suggests that meta-research enhances scientific rigour and contributes to broader societal outcomes by promoting a culture of evidence-based practice and informed health policies. Interestingly, the lack of association with research and development expenditure highlights a distinctive aspect of meta-research: its reliance on secondary data, which demands fewer resources than primary or experimental studies. This attribute makes meta-research a viable strategy for resource-limited settings, enabling impactful scientific contributions without the prohibitive costs typically associated with high-resource research projects.

The positive correlation between meta-research output and clinical trial completions suggests that meta-research activities may stimulate improved study designs and heightened scrutiny of methodological quality in clinical studies. This relationship is vital in an era where reproducibility and transparency are increasingly scrutinised in scientific practice. The novel insight lies in identifying meta-research as a strategic pathway for low- and middle-income countries to elevate their scientific and human development indicators without extensive financial investment. Encouraging meta-research growth in these regions can bridge scientific disparities and foster a more equitable global research landscape. Consequently, policy interventions to support meta-research infrastructures in underfunded regions could play a transformative role in advancing local and global health research outcomes. Some limitations of the study include that the country of the first author could not be identified in approximately one-third of the originally obtained articles, only a few indicators were included in the regression models, and the possible interactions or influences between them were not assessed. Future research should focus on studying the impact of MP using a wider range of indicators that cover major areas of health research and related fields.

Supplementary Materials

The following supporting information can be downloaded at https://www.mdpi.com/article/10.3390/publications13010007/s1, Table S1: Total number of publications and citations of the top 20 countries with the most publications; Table S2: Total number of publications and citations of the top 20 journals with the most publications; Table S3: Total number of publications and citations of the top 20 publishing groups with the most publications.

Author Contributions

I.D.L.-M.: Conceptualisation, methodology, investigation, data curation, formal analysis, writing—original draft preparation, and writing review and editing. D.A.H.-P.: Conceptualisation, methodology, investigation, data curation, formal analysis, writing—original draft preparation, and writing review and editing. O.F.-M.: Conceptualisation, methodology, investigation, data curation, formal analysis, writing—original draft preparation, and writing review and editing. Y.A.P.-J.: Conceptualisation, methodology, investigation, data curation, formal analysis, writing—original draft preparation, and writing review and editing. V.B.: Investigation, data curation, formal analysis, writing—original draft preparation, and writing review and editing. All authors have read and agreed to the published version of the manuscript.

Funding

Internal funds for research strengthening from Universidad Simón Bolívar, Vicerrectoría de Investigación, Extensión e Innovación, Barranquilla, Colombia.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Materials. Further inquiries can be directed to the corresponding authors.

Conflicts of Interest

The authors declare no conflicts of interest.

References

Beek, D., Gans, J., McIntyre, P., & Prasad, K. (2008). Cochrane review: Corticosteroids for acute bacterial meningitis. Evidence-Based Child Health: A Cochrane Review Journal, 3(2), 405–450. [Google Scholar] [CrossRef]
Bornmann, L., & Mutz, R. (2015). Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references. Journal of the Association for Information Science and Technology, 66(11), 2215–2222. [Google Scholar] [CrossRef]
Boury, H., Albert, M., Chen, R. H., Chow, J. C., DaCosta, R., Hoffman, M. M., Keshavarz, B., Kontos, P., McAndrews, M. P., Protze, S., & Gagliardi, A. R. (2023). Exploring the merits of research performance measures that comply with the San Francisco declaration on research assessment and strategies to overcome barriers of adoption: Qualitative interviews with administrators and re-searchers. Health Research Policy and Systems, 21(1), 43. [Google Scholar] [CrossRef] [PubMed]
Brouwer, M. C., McIntyre, P., Prasad, K., & van de Beek, D. (2015). Corticosteroids for acute bacterial meningitis. Cochrane Database of Systematic Reviews, 2018(11), CD004405. [Google Scholar] [CrossRef] [PubMed]
Djokoto, G. J. (2022). The investment development path and human development: Is there a nexus? Research in Globalization, 4, 100079. [Google Scholar] [CrossRef]
Fabiano, N., Gupta, A., Fiedorowicz, J. G., & Solmi, M. (2024). On the value of meta-research for early career researchers: A com-mentary. JCPP Advances, 4(2), e12235. [Google Scholar] [CrossRef]
Faggion, C. M., Jr. (2021). The importance and need of more meta-research studies in medical ethics journals. Accountability in Research, 28(2), 125–131. [Google Scholar] [CrossRef]
Fanelli, D., Costas, R., & Ioannidis, J. P. A. (2017). Meta-assessment of bias in science. Proceedings of the National Academy of Sciences, 114(14), 3714–3719. [Google Scholar] [CrossRef]
González-Márquez, R., Schmidt, L., Schmidt, B. M., Berens, P., & Kobak, D. (2024). The landscape of biomedical research. Patterns, 5(6), 100968. [Google Scholar] [CrossRef] [PubMed]
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. [Google Scholar] [CrossRef] [PubMed]
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. [Google Scholar] [CrossRef]
Ioannidis, J. P. A. (2008). Why most discovered true associations are inflated. Epidemiology, 19(5), 640–648. [Google Scholar] [CrossRef]
Ioannidis, J. P. A. (2010). Meta-research: The art of getting it wrong. Research Synthesis Methods, 1(3–4), 169–184. [Google Scholar] [CrossRef]
Ioannidis, J. P. A. (2011). Fund people not projects. Nature, 477(7366), 529–531. [Google Scholar] [CrossRef] [PubMed]
Ioannidis, J. P. A. (2014). How to make more published research true. PLoS Medicine, 11(10), e1001747. [Google Scholar] [CrossRef]
Ioannidis, J. P. A., Fanelli, D., Dunne, D. D., & Goodman, S. N. (2015). Meta-research: Evaluation and improvement of research methods and practices. PLoS Biology, 13(10), e1002264. [Google Scholar] [CrossRef] [PubMed]
Ioannidis, J., Samarel, M., Lau, J., & Drapkin, M. (1994). Risk of gastrointestinal bleeding from dexamethasone in children with bacterial meningitis. The Lancet, 343(8900), 792. [Google Scholar] [CrossRef]
Larsen, P. O., & von Ins, M. (2010). The rate of growth in scientific publication and the decline in coverage provided by science citation index. Scientometrics, 84(3), 575–603. [Google Scholar] [CrossRef] [PubMed]
Lozada-Martinez, I. D., Bolaño-Romero, M. P., Picón-Jaimes, Y. A., Moscote-Salazar, L. R., & Narvaez-Rojas, A. R. (2022). Quality or quantity? Questions on the growth of global scientific production. International Journal of Surgery, 105, 106862. [Google Scholar] [CrossRef] [PubMed]
Lozada-Martinez, I. D., Ealo-Cardona, C. I., Marrugo-Ortiz, A. C., Picón-Jaimes, Y. A., Cabrera-Vargas, L. F., & Narvaez-Rojas, A. R. (2023). Meta-research studies in surgery: A field that should be encouraged to assess and improve the quality of surgical evidence. International Journal of Surgery, 109(6), 1823–1824. [Google Scholar] [CrossRef] [PubMed]
Lozada-Martinez, I. D., Hernandez-Paez, D. A., Velasco, I. P., Guevara, D. M., & Liscano, Y. (2024a). Meta-research in geriatric surgery: Improving the quality of surgical evidence for older persons in a multidimensional-scale research field. Journal of Clinical Medicine, 13(18), 5441. [Google Scholar] [CrossRef]
Lozada-Martinez, I. D., Lozada-Martinez, L. M., & Fiorillo-Moreno, O. (2024b). Leiden manifesto and evidence-based research: Are the appropriate standards being used for the correct evaluation of pluralism, gaps and relevance in medical research? Journal of the Royal College of Physicians of Edinburgh, 54(1), 4–6. [Google Scholar] [CrossRef]
Lund, H., Juhl, C. B., Nørgaard, B., Draborg, E., Henriksen, M., Andreasen, J., Christensen, R., Nasser, M., Ciliska, D., Clarke, M., Tugwell, P., Martin, J., Blaine, C., Brunnhuber, K., & Robinson, K. A. (2020a). Evidence-Based Research Series-Paper 2: Using an Evidence-Based Research approach before a new study is conducted to ensure value. Journal of Clinical Epidemiology, 129, 158–166. [Google Scholar] [CrossRef] [PubMed]
Lund, H., Juhl, C. B., Nørgaard, B., Draborg, E., Henriksen, M., Andreasen, J., Christensen, R., Nasser, M., Ciliska, D., Tugwell, P., Clarke, M., Blaine, C., Martin, J., Ban, J., Brunnhuber, K., & Robinson, K. A. (2020b). Evidence-Based Research Series-Paper 3: Using an Evidence-Based Research approach to place your results into context after the study is performed to ensure usefulness of the conclusion. Journal of Clinical Epidemiology, 129, 167–171. [Google Scholar] [CrossRef]
McIntyre, P. B. (1997). Dexamethasone as adjunctive therapy in bacterial meningitis. A meta-analysis of randomized clinical trials since 1988. JAMA, 278(11), 925–931. [Google Scholar] [CrossRef]
Qin, Z., Zhang, H., & Zhao, B. (2023). Impact of R&D investment and network penetration on human development: Evidence from China. Social Indicators Research, 167(1–3), 1–25. [Google Scholar] [CrossRef]
Robinson, K. A., Brunnhuber, K., Ciliska, D., Juhl, C. B., Christensen, R., & Lund, H. (2021). Evidence-based research series-paper 1: What evidence-based research is and why is it important? Journal of Clinical Epidemiology, 129, 151–157. [Google Scholar] [CrossRef]
Rocha, G. M., & Grisolia, C. K. (2018). Why pesticides with mutagenic, carcinogenic and reproductive risks are registered in Brazil. Developing World Bioethics, 19(3), 148–154. [Google Scholar] [CrossRef] [PubMed]
Vardakas, K. Z., Matthaiou, D. K., & Falagas, M. E. (2009). Adjunctive dexamethasone therapy for bacterial meningitis in adults: A meta-analysis of randomised controlled trials. European Journal of Neurology, 16(6), 662–673. [Google Scholar] [CrossRef]
Wu, Y., Howarth, M., Zhou, C., Hu, M., & Cong, W. (2019). Reporting of ethical approval and informed consent in clinical research published in leading nursing journals: A retrospective observational study. BMC Medical Ethics, 20(1), 1–10. [Google Scholar] [CrossRef]
Zhang, L., Rousseau, R., & Sivertsen, G. (2017). Science deserves to be judged by its contents, not by its wrapping: Revisiting Seglen’s work on journal impact and research evaluation. PLoS ONE, 12(3), e0174205. [Google Scholar] [CrossRef] [PubMed]

Figure 1. Global distribution of meta-research-related publications in biomedicine by country. Each country is shaded according to the total number of publications, with darker shades representing a higher volume. The United States leads in meta-research output, as indicated by the darkest red shading, with 2447 publications. Other countries with high publication volumes, shown in varying shades of blue, include the United Kingdom, Canada, and several European and Asian countries. In contrast, countries shaded in light grey lack reported data in this dataset.

Figure 2. Global and categorical analysis of meta-research-related publications in biomedicine. (a) World distribution of meta-research publications by country from 1948 to 2024. (b) Total citations by country for meta-research publications indicate the impact of each country’s contributions over the same period. (c) Distribution of publication types within meta-research, categorised into original research articles, reviews, editorials, and other document types, illustrating the predominant publication formats for the top five countries with the highest publication volumes. (d) Journals with the highest number of meta-research publications from 1970 to 2024. (e) Publishing groups with the highest number of meta-research publications from 1981 to 2024.

Figure 3. Visualisation of the regression model between MP and global health, science, and human development indicators. (a) Association between the logarithm of meta-research publications count and the summed HDI, with fitted regression line and confidence band. (b) Scatter plot with regression between the meta-research articles’ logarithm and completed clinical trials’ logarithm. (c) Plot of the relationship between the logarithm of meta-research articles (with a one-year lag) and the logarithm of completed clinical trials. HDI: Human Development Index. MP: Meta-Research-related Publications in Biomedicine.

Table 1. Summary of regression analyses examining the relationship between log-transformed meta-research publications count and three outcome indicators: HDI, R&D expenditure (% of GDP), and completed clinical trials.

Analysis Type	Outcome	Variable	Estimate	SE	t-Value	p-Value	R²
Simple Regression	HDI	Intercept	29.8701	3.4650	8.621	<0.001	-
		log(Publications)	6.5454	0.7194	9.098	<0.001	0.8422
Simple Regression	R&D Expenditure	Intercept	3.8426	0.5409	7.104	<0.001	-
		log(Publications)	0.0531	0.1092	0.486	0.631	0.0093
Simple Regression	Clinical Trials	Intercept	0.5378	0.8600	0.625	0.54	-
		log(Publications)	1.6056	0.1586	10.120	<0.001	0.8505
Lagged regression (1 yr)	Clinical Trials	Intercept	1.6878	0.8454	1.997	0.062	-
		Lagged log(Publications)	1.4194	0.1567	9.058	<0.001	0.8183

HDI: Human Development Index; R&D: Research and Development; SE: Standard Error.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Lozada-Martinez, I.D.; Hernandez-Paz, D.A.; Fiorillo-Moreno, O.; Picón-Jaimes, Y.A.; Bermúdez, V. Meta-Research in Biomedical Investigation: Gaps and Opportunities Based on Meta-Research Publications and Global Indicators in Health, Science, and Human Development. Publications 2025, 13, 7. https://doi.org/10.3390/publications13010007

AMA Style

Lozada-Martinez ID, Hernandez-Paz DA, Fiorillo-Moreno O, Picón-Jaimes YA, Bermúdez V. Meta-Research in Biomedical Investigation: Gaps and Opportunities Based on Meta-Research Publications and Global Indicators in Health, Science, and Human Development. Publications. 2025; 13(1):7. https://doi.org/10.3390/publications13010007

Chicago/Turabian Style

Lozada-Martinez, Ivan David, David A. Hernandez-Paz, Ornella Fiorillo-Moreno, Yelson Alejandro Picón-Jaimes, and Valmore Bermúdez. 2025. "Meta-Research in Biomedical Investigation: Gaps and Opportunities Based on Meta-Research Publications and Global Indicators in Health, Science, and Human Development" Publications 13, no. 1: 7. https://doi.org/10.3390/publications13010007

APA Style

Lozada-Martinez, I. D., Hernandez-Paz, D. A., Fiorillo-Moreno, O., Picón-Jaimes, Y. A., & Bermúdez, V. (2025). Meta-Research in Biomedical Investigation: Gaps and Opportunities Based on Meta-Research Publications and Global Indicators in Health, Science, and Human Development. Publications, 13(1), 7. https://doi.org/10.3390/publications13010007

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu