Scientific papers can take months or even years to get published in an academic journal. Preprints, on the other hand, can be deposited into online repositories and are typically made available within a day or two of submission. So when COVID-19 started spreading rapidly across the globe, scientists worldwide turned to preprint servers to share their findings at the pace needed to address the crisis.
Preprints have long been a fixture in the physical sciences. The longest-standing preprint server, arXiv, was launched for the physics community in 1991. The biomedical sciences, however, took much longer to embrace preprints. The first preprint server for biology, bioRxiv, was established in 2013, and medRxiv, a preprint server for the health sciences, came to life just a few years ago, in 2019. Today there are dozens of preprints servers that cover many different disciplines—and the pandemic has accelerated their use.
One study found that 28.7 percent of COVID-19 preprints were cited in at least one news article, compared to 1 percent of preprints unrelated to the pandemic.
“This is the first global health crisis in which preprints have played a major role in disseminating information rapidly,” says Jessica Polka, the executive director of ASAPbio (Accelerating Science and Publication in biology), a nonprofit that advocates for open and innovative communication in the life sciences. “Even beyond the academic research community, I think many people learned about preprints for the first time. . . . [Preprints have] entered into the consciousness of both scientific and nonscientific communities.”
The use of preprints was on the rise before COVID-19, but the global health crisis has given them a huge boost in popularity, notes Gaetan Burgio, a geneticist at the Australian National University and one of bioRxiv’s affiliates—members of the scientific community who screen submissions and act as advocates for the service. But even as posting preprints becomes more common, certain sectors of the scientific community remain hesitant, pointing to issues such as their lack of peer review. Likewise, while some funding agencies and journals adopt more preprint-friendly policies, others have not. Last month, for example, the Australian Research Council (ARC) caused an uproar among scientists on social media when it barred grant applications that cited preprints.
See “Opinion: The Rise of Preprints Is No Cause for Alarm”
“The pandemic has probably heightened the response that [the ARC has] received [after] making that decision, because so many people realize just how beneficial [preprints] can be,” says Jonathon Coates, a postdoctoral researcher at the William Harvey Research Institute in the UK. “I think a couple of years ago we would have still had a reaction, but it wouldn't have been quite as strong.”
A flood of preprints
Preprints have made up a large chunk of the COVID-19 literature to date. According to a study by Coates, Polka, and their colleagues, of the approximately 125,000 COVID-19–related scientific articles published in the first 10 months of the pandemic, more than 30,000 (around one-quarter) were preprints. “Normally you would find around three or four percent of the biomedical literature as preprints each year,” Coates says. “A lot of people have actually been turning to them for the first time.”
The largest number of COVID-19 preprints appeared on medRxiv. Researchers also posted their work onto more than a dozen other preprint servers, including bioRxiv, arXiv, and SSRN, a server for studies in the social sciences.
© BRYAN SATALINO
During the pandemic, journalists and policymakers have also paid increased attention to preprints. Coates and Polka’s study found that 28.7 percent of COVID-19 preprints were cited in at least one news article, compared to 1 percent of preprints unrelated to the pandemic. They also discovered that preprints were cited in 52 of 81 COVID-19 policy documents from three large health authorities—the European Centre for Disease Prevention and Control (ECDC), the United Kingdom Parliamentary Office of Science and Technology (UK POST), and the World Health Organization (WHO).
Sarvenaz Sarabipour, a systems biologist at Johns Hopkins University, has seen the benefits of being able to rapidly disseminate research during the pandemic: a preprint Sarabipour and her colleagues published on bioRxiv last spring kick-started policy changes in advance of the study’s publication in Nature Human Behavior in March of this year.
Sarabipour’s group had examined more than 200 academic meetings worldwide and made suggestions for how to improve facets such as inclusivity and sustainability. When the team posted the preprint in early April 2020, “it was exactly the time when researchers were thinking about going online for conferences because of the pandemic,” Sarabipour recalls. “A lot of people contacted us and said, ‘I read your article and actually implemented these changes to virtual conferences in 2020 and 2021.’”
To preprint or not to preprint?
Some in the academic community remain skeptical of preprints’ benefits, however, and point to potential drawbacks. “The problems people have generally have with preprints is that they’re not peer reviewed,” Coates says. “And the assumption is they speed up science, and therefore the science they’re doing is rushed.”
I think that the overall trend is in a pro-preprint direction.
—Jessica Polka, ASAPbio
Indeed, some members of the scientific community have recently noted that the speedy nature of preprints, and their lack of peer review, could lead to the spread of misinformation. As one example, a widely publicized preprint claimed that there were signs of HIV “insertions” within the SARS-CoV-2 genome. Although it was quickly withdrawn, that preprint became fodder for conspiracy theories that the coronavirus was bioengineered. “[I]nstances such as this one described showcase the need for caution when acting upon the science put forth by any one preprint,” the authors of a commentary in The Lancet Global Health wrote last spring.
Swapnil Hiremath, a nephrologist and associate professor at the University of Ottawa in Canada, agrees that bad science gets published in preprints. There is “a lot of garbage that comes out,” he says. However, he adds, “one can argue that the peer reviewed literature is similarly flawed—it also has a lot of garbage.” According to a list kept by Retraction Watch, a total of 137 COVID-19–related manuscripts have been taken down by journals or preprint servers during the pandemic—and only 22 of those have been preprints.
While most preprints never make it to the published literatures, some studies suggest that for those that do, there is minimal difference in the quality between the preprint and published versions. Coates, Polka, and their colleagues compared preprints to their published versions. In a study posted on bioRxiv earlier this year, the team found that among preprints that were posted between January and April 2020 and published in a peer-reviewed journal in that same time period, the conclusions of abstracts usually did not change. Moreover, the figures and tables only underwent minor changes between the preprint and peer-reviewed manuscripts, suggesting that few additional experiments or analyses were carried out. A similar study by another group, published in a peer-reviewed journal last December, also found little difference between 2016 preprints and their peer-reviewed versions.
In response to the growing usage of preprints, many funding agencies, such as The Wellcome Trust, the European Research Council, and the National Institutes of Health, have changed their policies to accept—and in some cases, encourage—the use of preprints in grant applications. The ARC’s decision to bar preprints was “the first policy that I had heard of from a funder that discourages the citation of preprints,” Polka says. Similarly with journals, while there are some that have such anti-preprint policies, “I think that the overall trend is in a pro-preprint direction,” Polka tells The Scientist.
Whether these practices will remain once the pandemic is over remains to be seen. Hiremath and others say they hope that preprints are here to stay, since even prior to the flood of papers that came with COVID-19, they saw issues with the peer-review system, including its lack of transparency and the sometimes-false sense of validity it provides. “I think we do need to rethink what peer review is and how we validate science or how we decide what science is good or not,” Coates says. “Because the current system—I don’t think works.”
