We revisit the results of the recent Reproducibility Project: Psychology by the Open Science Collaboration. We compute Bayes factors—a quantity that can be used to express comparative evidence for an hypothesis but also for the null hypothesis—for a large subset ( N = 72) of the original papers and their corresponding replication attempts. In our computation, we take into account the likely scenario that publication bias had distorted the originally published results. Overall, 75% of studies gave qualitatively similar results in terms of the amount of evidence provided. However, the evidence was often weak (i.e., Bayes factor < 10). The majority of the studies (64%) did not provide strong evidence for either the null or the alternative hypothesis in either the original or the replication, and no replication attempts provided strong evidence in favor of the null. In all cases where the original paper provided strong evidence but the replication did not (15%), the sample size in the replication was smaller than the original. Where the replication provided strong evidence but the original did not (10%), the replication sample size was larger. We conclude that the apparent failure of the Reproducibility Project to replicate many target effects can be adequately explained by overestimation of effect sizes (or overestimation of evidence against the null hypothesis) due to small sample sizes and publication bias in the psychological literature. We further conclude that traditional sample sizes are insufficient and that a more widespread adoption of Bayesian methods is desirable.

Their use and platforms require greater scrutiny Preprints-manuscripts that have not undergone peer review-were first embraced in physics, catalysed by the creation in the early 1990s of arXiv.org, an open online repository for scholarly papers.1 It was not until 2013 that similar initiatives were embraced by the biological and then medical sciences,2 and novel publishing platforms continue to emerge. Some commentators believe the potential for harm is outweighed by the benefits,134 but others have raised specific concerns regarding medical preprints and mitigating the risk of harm to the public.2 These discussions need to be revisited in the context of the covid-19 pandemic, which has been accompanied by an explosion of preprint publications. An analysis focusing on studies estimating the R of SARS-CoV-2 drew attention to the powerful role of preprints in shaping global discourse about covid-19 transmissibility. While showing the benefits that preprints may confer when adopting a consensus based approach-where data is extracted from multiple studies to observe trends and obtain an average with or without the exclusion of outliers-the authors also identify risks-matters of credibility and misinformation, both intentional and unintentional5-which may be increased where there are vested interests involved. Notably, two linked preprint publications examining the association between smoking and covid-19,67 which were widely disseminated before …

Context Convergent analyses in different disciplines support the use of the Percentage of Papers by the Most Prolific author (PPMP) as a red flag to identify journals that can be suspected of questionable editorial practices. We examined whether this index, complemented by the Gini index, could be useful for identifying cases of potential editorial bias, using a large sample of biomedical journals. Methods We extracted metadata for all biomedical journals referenced in the National Library of Medicine, with any attributed Broad Subject Terms, and at least 50 authored (i.e. by at least one author) articles between 2015 and 2019, identifying the most prolific author (i.e. the person who signed the most papers in each particular journal). We calculated the PPMP and the 2015-2019 Gini index for the distribution of articles across authors. When the relevant information was reported, we also computed the median publication lag (time between submission and acceptance) for articles authored by any of the most prolific authors and that for articles not authored by prolific authors. For outlier journals, defined as a PPMP or Gini index above the 95th percentile of their respective distributions, a random sample of 100 journals was selected and described in relation to status on the editorial board for the most prolific author. Results 5 468 journals that published 4 986 335 papers between 2015 and 2019 were analysed. The PPMP 95th percentile was 10.6% (median 2.9%). The Gini index 95th percentile was 0.355 (median 0.183). Correlation between the two indices was 0.35 (95CI 0.33 to 0.37). Information on publication lag was available for 2 743 journals. We found that 277 journals (10.2%) had a median time lag to publication for articles by the most prolific author(s) that was shorter than 3 weeks, versus 51 (1.9%) journals with articles not authored by prolific author(s). Among the random sample of outlier journals, 98 provided information about their editorial board. Among these 98, the most prolific author was part of the editorial board in 60 cases (61%), among whom 25 (26% of the 98) were editors-in-chief. Discussion In most journals publications are distributed across a large number of authors. Our results reveal a subset of journals where a few authors, often members of the editorial board, were responsible for a disproportionate number of publications. The papers by these authors were more likely to be accepted for publication within 3 weeks of their submission. To enhance trust in their practices, journals need to be transparent about their editorial and peer review practices.