This post is by Sarah Greaves, Independent Consultant & Phil Hurst, Publisher, The Royal Society
We are pleased to provide this update to an interim report published in April 2021 regarding the COVID-19 Rapid Reviewer Initiative. This post summarises the group’s high level findings on our key aims. In a few weeks a more detailed analysis will be published.
As detailed here on the OASPA blog, the COVID-19 Rapid Reviewer Initiative was set up by a group of publishers and scholarly communications organisations in March 2020 in response to the rapid increase in submissions on COVID-19 and the rise of preprints and pressure on the peer review system overall.
The aims of the group were published in a Letter of Intent and aimed to address the following key elements:
Aim 1. Expand the pool of rapid reviewers and improve turnaround times
Aim 2. Encourage transfer of manuscripts between publishers
Aim 3. Encourage articles submitted to journals to have a preprint
Aim 4. Ensure that authors share the data
The success of Aim 1 in terms of recruiting rapid reviewers has been highlighted elsewhere and data from all the participating publishers now shows times to acceptance and publication were quicker for COVID-19 papers.
The data showed that the group received more than 13,000 COVID-19 related submissions between January 2020 and March 2021. Of these, around 16% were accepted. Most papers were submitted between May and June 2020 with some publishers seeing a second peak between November and December 2020. The acceptance rate for COVID-19 papers from the specific journals participating in the collaboration was lower than the historic baselines for all papers published by participating journals over the same time period.
Aim 2 has been discussed previously as not being successful as we did not see any transfers between journals involved in this initiative. Instead some authors elected to transfer reports to another journal within a publisher’s portfolio or simply resubmitted to another publisher without transfer of reports.
There is a clear reluctance by authors to transfer negative reviewer reports. We need to move away from the culture of acceptance or rejection after peer review to an open research infrastructure where reviews and author responses exist to help improve the work, rather than justify a publishing decision.
Regarding Aim 3, related to preprint publication, some of the publishers involved in the initiative altered their policies in line with this requirement from the Wellcome call but compliance was mixed. As part of the initiative, PREreview collated all COVID-19 related preprints and encouraged the rapid reviewers to review these (ASAPbio blog) with some limited success. The same was seen at Rapid Reviews COVID-19, an overlay journal run by MIT Press, which was also part of the initiative. While peer review of preprints is still a relatively new development, these statistics demonstrate the burgeoning interest in reviewing preprints as a complement to peer review carried out by journals.
Finally, in terms of Aim 4, related to data sharing, the group was mainly successful. The group worked with FAIRsharing (https://fairsharing.org) to ensure their policies were publicly registered and mandated Data Availability Statements (DAS). The result is a growing collection of policies, which are easily discoverable and more comparable.
After a long year working at speed and across numerous organisations we thank everyone who took part in the initiative and shared their data to help us analyse whether we achieved our stated aims. It’s clear that we did achieve some but have work to do on others.
In early December the Research on Research Institute (RoRI), in collaboration with a number of organisations in the COVID-19 Rapid Review Initiative, will publish a detailed report on some of the learnings from the initiative – so please look out for that report which will be communicated widely. Furthermore, on January 13 next year, OASPA will host a webinar, involving RORI, which will delve deeper into the report’s key findings. Information will soon be shared widely with the registration link.
For us a key outcome is that we learnt publishers could mobilise themselves to respond to a crisis and collaborate with other organisations to meet a global need. There is the opportunity to learn, not only from this initiative but from the many others during the pandemic, as to how we might change some of our scholarly publishing practices and respond more effectively in the future.