Canibais e Reis

Seguem-se vários artigos do Dr. John Ioannidis, Professor de Medicina da Tufts University School of Medicine e Professor de Epidemiologia na Harvard School of Public Health. É a primeira vez que eu vejo um investigador com um curriculum vitae com 91 páginas e também com mais de 400 entradas na Pubmed. Como não poderia deixar de ser, o seu artigo mais citado é o "Why most published research findings are false." Veja várias discussões sobre este polémico artigo aqui: 1, 2, 3, 4 e 5.

Eur J Clin Invest. 2010 Apr;40(4):285-7.

Who is afraid of reviewers’ comments? Or, why anything can be published and anything can be cited. (pdf)

Ioannidis JP, Tatsioni A, Karassa FB.

 

Eur J Clin Invest. 2010 Feb;40(2):172-82. Epub 2009 Dec 27.

Industry sponsorship and selection of comparators in randomized clinical trials. (pdf)

Lathyris DN, Patsopoulos NA, Salanti G, Ioannidis JP.
General Hospital George Papanikolaou, Thessaloniki, Greece.

Abstract

BACKGROUND: Most clinical trials on medical interventions are sponsored by the industry. The choice of comparators shapes the accumulated evidence. We aimed to assess how often major companies sponsor trials that involve only their own products. METHODS: Studies were identified by searching ClinicalTrials.gov for trials registered in 2006. We focused on randomized trials involving the 15 companies that had sponsored the largest number of registered trials in ClinicalTrials.gov in that period. RESULTS: Overall, 577 randomized trials were eligible for analysis and 82% had a single industry sponsor [89% (166/187) of the placebo-control trials, 87% (91/105) of trials comparing different doses or ways of administration of the same intervention, and 78% (221/285) of other active control trials]. The compared intervention(s) belonged to a single company in 67% of the trials (89%, 81% and 47% in the three categories respectively). All 15 companies strongly preferred to run trials where they were the only industry sponsor or even the only owner of the assessed interventions. Co-sponsorship typically reflected co-ownership of the same intervention by both companies. Head-to-head comparison of different active interventions developed by different companies occurred in only 18 trials with two or more industry sponsors. CONCLUSIONS: Each company generates a clinical research agenda that is strongly focused on its own products, while comparisons involving different interventions from different companies are uncommon. This diminishes the ability to understand the relative merits of different interventions for the same condition.

 

PLoS Med. 2009 Jul 21;6(7):e1000100. Epub 2009 Jul 21.

The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. (pdf)

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D.
Università di Modena e Reggio Emilia, Modena, Italy.

Abstract

Systematic reviews and meta-analyses are essential to summarize evidence relating to efficacy and safety of health care interventions accurately and reliably. The clarity and transparency of these reports, however, is not optimal. Poor reporting of systematic reviews diminishes their value to clinicians, policy makers, and other users.Since the development of the QUOROM (QUality Of Reporting Of Meta-analysis) Statement–a reporting guideline published in 1999–there have been several conceptual, methodological, and practical advances regarding the conduct and reporting of systematic reviews and meta-analyses. Also, reviews of published systematic reviews have found that key information about these studies is often poorly reported. Realizing these issues, an international group that included experienced authors and methodologists developed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) as an evolution of the original QUOROM guideline for systematic reviews and meta-analyses of evaluations of health care interventions.The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram. The checklist includes items deemed essential for transparent reporting of a systematic review. In this Explanation and Elaboration document, we explain the meaning and rationale for each checklist item. For each item, we include an example of good reporting and, where possible, references to relevant empirical studies and methodological literature. The PRISMA Statement, this document, and the associated Web site (http://www.prisma-statement.org/) should be helpful resources to improve reporting of systematic reviews and meta-analyses.

 

Open Med. 2009 May 26;3(2):e62-8.

The use of older studies in meta-analyses of medical interventions: a survey. (pdf)

Patsopoulos NA, Ioannidis JP.

Abstract

BACKGROUND: Evidence for medical interventions sometimes derives from data that are no longer up to date. These data can influence the outcomes of meta-analyses, yet do not always reflect current clinical practice. We examined the age of the data used in meta-analyses contained within systematic reviews of medical interventions, and investigated whether authors consider the age of these data in their interpretations. METHODS: From Issue 4, 2005, of the Cochrane Database of Systematic Reviews we randomly selected 10% of systematic reviews containing at least 1 meta-analysis. From this sample we extracted 1 meta-analysis per primary outcome. We calculated the number of years between the study’s publication and 2005 (the year that the systematic review was published), as well as the number of years between the study’s publication and the year of the literature search conducted in the study. We assessed whether authors discussed the implications of including less recent data, and, for systematic reviews containing meta-analyses of studies published before 1996, we calculated whether excluding the findings of those studies changed the significance of the outcomes. We repeated these calculations and assessments for 22 systematic reviews containing meta-analyses published in 6 high-impact general medical journals in 2005. RESULTS: For 157 meta-analyses (n = 1149 trials) published in 2005, the median year of the most recent literature search was 2003 (interquartile range [IQR] 2002-04). Two-thirds of these meta-analyses (103/157, 66%) involved no trials published in the preceding 5 years (2001-05). Forty-seven meta-analyses (30%) included no trials published in the preceding 10 years (1996-2005). In another 16 (10%), the statistical significance of the outcomes would have been different had the studies been limited to those published between 1996 and 2005, although in some cases this change in significance would have been due to loss of power. Only 12 (8%) of the meta-analyses discussed the potential implications of including older studies. Among the 22 meta-analyses considered in high-impact general medical journals, 2 included no studies published in the 5 years prior to the reference year (2005), and 18 included at least 1 study published before 1996. Only 4 meta-analyses discussed the implications of including older studies. INTERPRETATION: In most systematic reviews containing meta-analyses of evidence for health care interventions, very recent studies are rare. Researchers who conduct systematic reviews with meta-analyses, and clinicians who read the outcomes of these studies, should be made aware of the potential implications of including less recent data.

 

CMAJ. 2009 Oct 13;181(8):488-93. Epub 2009 Aug 4.

Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. (pdf)

Ioannidis JP.
Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece.

 

PLoS Med. 2008 Oct 7;5(10):e201.

Why current publication practices may distort science. (pdf)

Young NS, Ioannidis JP, Al-Ubaydli O.
Hematology Branch, National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, Maryland, United States of America.

 

PLoS One. 2008 Aug 28;3(8):e3081.

Systematic review of the empirical evidence of study publication bias and outcome reporting bias. (pdf)

Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, Ghersi D, Ioannidis JP, Simes J, Williamson PR.
Centre for Medical Statistics and Health Evaluation, University of Liverpool, Liverpool, United Kingdom.

Abstract

BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention. METHODOLOGY/PRINCIPAL FINDINGS: We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

 

Am J Epidemiol. 2008 Aug 15;168(4):374-83; discussion 384-90. Epub 2008 Jul 8.

Effect of formal statistical significance on the credibility of observational associations. (pdf)

Ioannidis JP.
Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.

Abstract

The author evaluated the implications of nominal statistical significance for changing the credibility of null versus alternative hypotheses across a large number of observational associations for which formal statistical significance (p < 0.05) was claimed. Calculation of the Bayes factor (B) under different assumptions was performed on 272 observational associations published in 2004-2005 and a data set of 50 meta-analyses on gene-disease associations (752 studies) for which statistically significant associations had been claimed (p < 0.05). Depending on the formulation of the prior, statistically significant results offered less than strong support to the credibility (B > 0.10) for 54-77% of the 272 epidemiologic associations for diverse risk factors and 44-70% of the 50 associations from genetic meta-analyses. Sometimes nominally statistically significant results even decreased the credibility of the probed association in comparison with what was thought before the study was conducted. Five of six meta-analyses with less than substantial support (B > 0.032) lost their nominal statistical significance in a subsequent (more recent) meta-analysis, while this did not occur in any of seven meta-analyses with decisive support (B < 0.01). In these large data sets of observational associations, formal statistical significance alone failed to increase much the credibility of many postulated associations. Bayes factors may be used routinely to interpret "significant" associations.

 

Bull NYU Hosp Jt Dis. 2008;66(2):135-9.

Some main problems eroding the credibility and relevance of randomized trials. (pdf)

Ioannidis JP.
Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece 45110.

Abstract

Randomized trials are an excellent research design with major advantages. However, randomized trials are not immune to biases, and inferences from them may be sometimes flawed or irrelevant. The present review addresses, in brief, some of the major threats to the credibility and relevance of the results of clinical trials: power problems, biases affecting internal validity (poor design, conduct, and analysis), biases affecting the total randomized evidence on a specific topic (publication bias and selective outcome and analysis reporting bias), lack of relevance, poor generalizability, and biases in the interpretation of the results.

 

JAMA. 2007 Dec 5;298(21):2517-26.

Persistence of contradicted claims in the literature. (pdf)

Tatsioni A, Bonitsis NG, Ioannidis JP.
Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.

Abstract

CONTEXT: Some research findings based on observational epidemiology are contradicted by randomized trials, but may nevertheless still be supported in some scientific circles. OBJECTIVES: To evaluate the change over time in the content of citations for 2 highly cited epidemiological studies that proposed major cardiovascular benefits associated with vitamin E in 1993; and to understand how these benefits continued being defended in the literature, despite strong contradicting evidence from large randomized clinical trials (RCTs). To examine the generalizability of these findings, we also examined the extent of persistence of supporting citations for the highly cited and contradicted protective effects of beta-carotene on cancer and of estrogen on Alzheimer disease. DATA SOURCES: For vitamin E, we sampled articles published in 1997, 2001, and 2005 (before, early, and late after publication of refuting evidence) that referenced the highly cited epidemiological studies and separately sampled articles published in 2005 and referencing the major contradicting RCT (HOPE trial). We also sampled articles published in 2006 that referenced highly cited articles proposing benefits associated with beta-carotene for cancer (published in 1981 and contradicted long ago by RCTs in 1994-1996) and estrogen for Alzheimer disease (published in 1996 and contradicted recently by RCTs in 2004). DATA EXTRACTION: The stance of the citing articles was rated as favorable, equivocal, and unfavorable to the intervention. We also recorded the range of counterarguments raised to defend effectiveness against contradicting evidence. RESULTS: For the 2 vitamin E epidemiological studies, even in 2005, 50% of citing articles remained favorable. A favorable stance was independently less likely in more recent articles, specifically in articles that also cited the HOPE trial (odds ratio for 2001, 0.05 [95% confidence interval, 0.01-0.19; P < .001] and the odds ratio for 2005, 0.06 [95% confidence interval, 0.02-0.24; P < .001], as compared with 1997), and in general/internal medicine vs specialty journals. Among articles citing the HOPE trial in 2005, 41.4% were unfavorable. In 2006, 62.5% of articles referencing the highly cited article that had proposed beta-carotene and 61.7% of those referencing the highly cited article on estrogen effectiveness were still favorable; 100% and 96%, respectively, of the citations appeared in specialty journals; and citations were significantly less favorable (P = .001 and P = .009, respectively) when the major contradicting trials were also mentioned. Counterarguments defending vitamin E or estrogen included diverse selection and information biases and genuine differences across studies in participants, interventions, cointerventions, and outcomes. Favorable citations to beta-carotene, long after evidence contradicted its effectiveness, did not consider the contradicting evidence. CONCLUSION: Claims from highly cited observational studies persist and continue to be supported in the medical literature despite strong contradictory evidence from randomized trials.

 

PLoS Clin Trials. 2006 Nov 17;1(7):e36.

Evolution and translation of research findings: from bench to where? (pdf)

Ioannidis JP.
Department of Hygiene and Epidemiology, School of Medicine, University of Ioannina, Ioannina, Greece.

Abstract

The credibility and replication of research findings evolve over time, as data accumulate. However, translation of postulated research promises to real-life biomedical applications is uncommon. In some fields of research, we may observe diminishing effects for the strength of research findings and rapid alternations of exaggerated claims and extreme contradictions–the "Proteus Phenomenon." While these phenomena are probably more prominent in the basic sciences, similar manifestations have been documented even in clinical trials and they may undermine the credibility of clinical research. Significance-chasing bias may be in part responsible, but the greatest threat may come from the poor relevance and scientific rationale and thus low pre-study odds of success of research efforts. Given that we currently have too many research findings, often with low credibility, replication and rigorous evaluation become as important as or even more important than discovery. Credibility, replication, and translation are all desirable properties of research findings, but are only modestly correlated. In this essay, I discuss some of the evidence (or lack thereof) for the process of evolution and translation of research findings, with emphasis on the biomedical sciences.

 

Cancer Epidemiol Biomarkers Prev. 2006 Jan;15(1):186.

Journals should publish all "null" results and should sparingly publish "positive" results. (pdf)

Ioannidis JP.
Comment on: Cancer Epidemiol Biomarkers Prev. 2004 Dec;13(12):1985-6.

 

PLoS Med. 2007 Jun;4(6):e215.

Why most published research findings are false: author’s reply to Goodman and Greenland. (pdf)

Ioannidis JP.
Comment on: PLoS Med. 2007 Apr;4(4):e168.

 

PLoS Med. 2005 Aug;2(8):e124. Epub 2005 Aug 30.

Why most published research findings are false. (pdf)

Ioannidis JP.
Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.

Comment in: PLoS Med. 2007 Apr;4(4):e168. PLoS Med. 2005 Nov;2(11):e395. PLoS Med. 2005 Nov;2(11):e386; author reply e398. PLoS Med. 2005 Nov;2(11):e361. PLoS Med. 2005 Aug;2(8):e272.

Abstract

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

 

JAMA. 2005 Jul 13;294(2):218-28.

Contradicted and initially stronger effects in highly cited clinical research. (pdf)

Ioannidis JP.
Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.

Comment in: JAMA. 2005 Dec 7;294(21):2695; author reply 2696.JAMA. 2005 Dec 7;294(21):2695; author reply 2696.JAMA. 2005 Dec 7;294(21):2695-6; author reply 2696.

Abstract

CONTEXT: Controversy and uncertainty ensue when the results of clinical research on the effectiveness of interventions are subsequently contradicted. Controversies are most prominent when high-impact research is involved. OBJECTIVES: To understand how frequently highly cited studies are contradicted or find effects that are stronger than in other similar studies and to discern whether specific characteristics are associated with such refutation over time. DESIGN: All original clinical research studies published in 3 major general clinical journals or high-impact-factor specialty journals in 1990-2003 and cited more than 1000 times in the literature were examined. MAIN OUTCOME MEASURE: The results of highly cited articles were compared against subsequent studies of comparable or larger sample size and similar or better controlled designs. The same analysis was also performed comparatively for matched studies that were not so highly cited. RESULTS: Of 49 highly cited original clinical research studies, 45 claimed that the intervention was effective. Of these, 7 (16%) were contradicted by subsequent studies, 7 others (16%) had found effects that were stronger than those of subsequent studies, 20 (44%) were replicated, and 11 (24%) remained largely unchallenged. Five of 6 highly-cited nonrandomized studies had been contradicted or had found stronger effects vs 9 of 39 randomized controlled trials (P = .008). Among randomized trials, studies with contradicted or stronger effects were smaller (P = .009) than replicated or unchallenged studies although there was no statistically significant difference in their early or overall citation impact. Matched control studies did not have a significantly different share of refuted results than highly cited studies, but they included more studies with "negative" results. CONCLUSIONS: Contradiction and initially stronger effects are not unusual in highly cited research of clinical interventions and their outcomes. The extent to which high citations may provoke contradictions and vice versa needs more study. Controversies are most common with highly cited nonrandomized studies, but even the most highly cited randomized trials may be challenged and refuted over time, especially small ones.

 

JAMA. 2005 May 18;293(19):2362-6.

Relative citation impact of various study designs in the health sciences. (pdf)

Patsopoulos NA, Analatos AA, Ioannidis JP.
Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.

Abstract

CONTEXT: The relative merits of various study designs and their placement in hierarchies of evidence are often discussed. However, there is limited knowledge about the relative citation impact of articles using various study designs. OBJECTIVE: To determine whether the type of study design affects the rate of citation in subsequent articles. DESIGN AND SETTING: We measured the citation impact of articles using various study designs–including meta-analyses, randomized controlled trials, cohort studies, case-control studies, case reports, nonsystematic reviews, and decision analysis or cost-effectiveness analysis–published in 1991 and in 2001 for a sample of 2646 articles. MAIN OUTCOME MEASURE: The citation count through the end of the second year after the year of publication and the total received citations. RESULTS: Meta-analyses received more citations than any other study design both in 1991 (P<.05 for all comparisons) and in 2001 (P<.001 for all comparisons) and both in the first 2 years and in the longer term. More than 10 citations in the first 2 years were received by 32.4% of meta-analyses published in 1991 and 43.6% of meta-analyses published in 2001. Randomized controlled trials did not differ significantly from epidemiological studies and nonsystematic review articles in 1991 but clearly became the second-cited study design in 2001. Epidemiological studies, nonsystematic review articles, and decision and cost-effectiveness analyses had relatively similar impact; case reports received negligible citations. Meta-analyses were cited significantly more often than all other designs after adjusting for year of publication, high journal impact factor, and country of origin. When limited to studies addressing treatment effects, meta-analyses received more citations than randomized trials. CONCLUSION: Overall, the citation impact of various study designs is commensurate with most proposed hierarchies of evidence.

 

BMJ. 2004 Dec 18;329(7480):1436-40.

The Decameron of poor research. (pdf)

Berger VW, Ioannidis JP.
Biometry Research Group, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA.

 

Am J Epidemiol. 2001 Nov 1;154(9):873-80.

Effect of early patient enrollment on the time to completion and publication of randomized controlled trials. (pdf)

Haidich AB, Ioannidis JP.
Clinical Trials and Evidence-based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.

Abstract

The authors evaluated whether early enrollment affects the significance of the results and the time to completion and publication of randomized controlled trials. Seventy-seven efficacy randomized controlled trials (total enrollment, 28,992 patients) initiated by the Acquired Immunodeficiency Syndrome Clinical Trials Group between 1986 and 1996 were evaluated. After adjustment for target sample size, for each 10-fold increase in the first-month accrual, the odds of a trial reaching statistically significant results increased 2.8-fold (p = 0.040). The relative enrollment during the first month over target sample size (hazard ratio (HR) = 1.40 per 10 percent increase, p = 0.004) and masking (HR = 1.78 for double-blind vs. single or unblinded studies, p = 0.031) were the major predictors of faster completion. Rapid early accrual (HR = 1.09 per 10 additional patients accrued the first month, p = 0.011) and statistical significance in favor of an experimental arm (HR = 2.47, p = 0.004) independently predicted faster publication. Early enrollment is a strong predictor of whether a study will reach formal statistical significance, and it can offer predictive information on the time needed to complete the study and publish its findings. Ongoing unpublished studies and their enrollment rates may need to be considered when interpreting the accumulated evidence.

 

BMJ. 2001 Apr 14;322(7291):879-80.

Any casualties in the clash of randomised and observational evidence? (pdf)

Ioannidis JP, Haidich AB, Lau J.

 

JAMA. 1998 Jan 28;279(4):281-6.

Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. (pdf)

Ioannidis JP.
HIV Research Branch, Division of AIDS, National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Md 20892, USA.

Comment in: JAMA. 1998 Jun 24;279(24):1952; author reply 1952-3.JAMA. 1998 Jan 28;279(4):319-20.JAMA. 1998 Jun 24;279(24):1952; author reply 1952-3.

Abstract

CONTEXT: Medical evidence may be biased over time if completion and publication of randomized efficacy trials are delayed when results are not statistically significant. OBJECTIVE: To evaluate whether the time to completion and the time to publication of randomized phase 2 and phase 3 trials are affected by the statistical significance of results and to describe the natural history of such trials. DESIGN: Prospective cohort of randomized efficacy trials conducted by 2 trialist groups from 1986 to 1996. SETTING: Multicenter trial groups in human immunodeficiency virus infection sponsored by the National Institutes of Health. PATIENTS: A total of 109 efficacy trials (total enrollment, 43708 patients). MAIN OUTCOME MEASURES: Time from start of enrollment to completion of follow-up and time from completion of follow-up to peer-reviewed publication assessed with survival analysis. RESULTS: The median time from start of enrollment to publication was 5.5 years and was substantially longer for negative trials than for results favoring an experimental arm (6.5 vs 4.3 years, respectively; P<.001; hazard ratio for time to publication for positive vs negative trials, 3.7; 95% confidence interval [CI], 1.8-7.7). This difference was mostly attributable to differences in the time from completion to publication (median, 3.0 vs 1.7 years for negative vs positive trials; P<.001). On average, trials with significant results favoring any arm completed follow-up slightly earlier than trials with nonsignificant results (median, 2.3 vs 2.5 years; P=.045), but long-protracted trials often had low event rates and failed to reach statistical significance, while trials that were terminated early had significant results. Positive trials were submitted for publication significantly more rapidly after completion than were negative trials (median, 1.0 vs 1.6 years; P=.001) and were published more rapidly after submission (median, 0.8 vs 1.1 years; P=.04). CONCLUSION: Among randomized efficacy trials, there is a time lag in the publication of negative findings that occurs mostly after the completion of the trial follow-up.