OASPA’s second statement following the article in Science entitled “Who’s Afraid of Peer Review?”

Since OASPA released its first response to the Science ‘Sting’ article published in October, the OASPA Board has been looking at the implications of the findings for the organisation and its members.

There has also been much discussion of the Science article, exploring the strengths and weaknesses of the exercise and associated data, along with the way that Science has presented this work to the media.  In general, we feel that this follow-up has served to emphasise two key points: the data provide useful insight into editorial practices at a number of substandard publishers; no conclusions can be drawn as to whether poor practices are more or less prevalent at subscription-based or open-access journals.

The work at OASPA has been conducted, in consultation with the OASPA board, by the membership subcommittee, comprising Caroline Sutton, Catriona MacCallum, Claire Redhead and Mark Patterson.

We have looked at the data concerning the publishers who were identified in the Science sting as having accepted and rejected the article.  Initially four publisher members were indicated as having accepted the article.  However one of those, African Journals Online (AJOL), was incorrectly indentified.  AJOL is not a publisher, but is a non-profit organisation that provides free online hosting and aggregation services to qualifying African journals.  This information has been corrected on the Science website.  AJOL has let OASPA know that the African Journal of Biomedical Research, published by Ibadan Biomedical Communications Group, was deemed in compliance with AJOL’s inclusion criteria when it applied to AJOL for free hosting but has not sent content for upload by AJOL since 2008. To date, accepted journals have not been re-assessed to determine if they are continuing to adhere to AJOL’s inclusion criteria. However, AJOL has been developing a process (which will be in place by the end of 2014) to address the need for ongoing quality assessment of partner journals.

Three further OASPA members publish journals that accepted the article: Hikari (Clinical and Experimental Medical Sciences), Dove Medical Press (Drug Design, Development and Therapy) and SAGE (Journal of International Medical Research).  For each of these publishers we have carefully reviewed the correspondence made available by Science, the journal’s editorial practices, and have contacted relevant individuals for further information.  We have been very grateful in all cases for prompt feedback.   Our key conclusion from these discussions is that there was a lack of sufficient rigour in editorial processes at all three of the journals in question, and that for Hikari and Dove the issues may extend wider than the single affected journal.  Given OASPA’s commitment to high standards in all aspects of open-access publishing, we are therefore reluctantly terminating the memberships of Hikari and Dove Medical Press.  We have indicated that we will be willing to reconsider a membership application but not before 12 months have elapsed.

For SAGE, we have placed their OASPA membership ‘under review’ for the next 6 months, at which point we will review the journal in question and readmit SAGE as a full member if there is evidence that its processes have been sufficiently strengthened.  The different course of action that we have taken with SAGE reflects several issues:  (1) the editorial process at the Journal of International Medical Research is very unusual and not typical of any of the other journals at SAGE; (2) although the Science journalist was told that the article had been accepted, there was in fact a remaining step in the process (a detailed technical edit by domain experts) that is likely to have identified the deficiencies in the work; (3) SAGE acted promptly to cease taking submissions to the journal as soon as the problem had been identified, issued a press release explaining the situation, and has been very open about the problems that have been uncovered; and (4) SAGE has a very strong and longstanding reputation for responsible and ethical publishing practices.

We have also reviewed the correspondence released by Science in relation to members of OASPA that rejected the article, with a view to identifying any lessons that can be learned.   The weakness that seems to account most frequently for the failure of the editorial process in the journals that accepted the fake article was a lack of expert editorial accountability for the key decision points: the assignment of reviewers; assessment of review comments and decision letter; and the final acceptance decision.  Robust editorial processes require an engaged editor (usually an experienced academic researcher, but occasionally a suitably qualified staff editor) to take responsibility for these key decision points.

Another important action that OASPA has therefore taken is to strengthen its own membership procedures. The existing process requires an applicant to complete a form about its organisation and its practices with respect to key areas including licensing, publication fees, organisational status and ownership, and editorial practices, in particular peer review.  We have now added more detailed questions about the editorial process to the application form, focusing on the key decision points and how these are conducted. The answers to these questions will be added to the page that we provide on the OASPA site about each member (eg http://oaspa.org/member-record-american-institute-of-physics/). If we have any uncertainty about the answers given, then we also contact editors who are listed by the journal to seek their views on the journal and the publisher in question.  (We have already been doing this over the past year.)

In conclusion, although we have unfortunately now terminated memberships as a result of the Science news article, positive outcomes have also arisen from its publication.  The article highlights the need for organisations such as OASPA and DOAJ to act as important signals of reliable and high-quality publishing operations, and both organisations are strengthening their selection criteria. There will always be poor practices in any field, and if evidence of poor practice is identified, action can and will be taken. We encourage anyone with an interest in the integrity of research communication to report concerns to OASPA and DOAJ. At OASPA in particular our aim is to provide help and support where we can, so that new and genuine participants have the opportunity to thrive and so that we can build on the tremendous strides in open-access publishing that have been achieved throughout the world over the past 15 years.

4 thoughts on “OASPA’s second statement following the article in Science entitled “Who’s Afraid of Peer Review?”

  1. Bohannon has rightly appreciated the decision of OASPA to terminate the membership of Dove Press and Hikari, as a result of his investigation. I also support that as a result of Bohannon investigation, DOAJ has removed 114 OA journals from its list. Very good. Weeding is always necessary. OASPA and DOAJ is taking action to correct its list. Nobody is perfect and revision is always necessary once a kind of ‘peer-review report’ is available. But does anybody know that J Beall has taken any action to correct his famous list? The sting operation and all related discussions on internet is inclined to highlight who failed in this experiment. It is not telling or highlighting about those publishers who passed this experiment but still occupy the seat in Beall’s famous list. I really hate this trend.
    Similarly, the results show that neither the Directory of Open Access Journals (DOAJ), nor Beall’s List are accurate in detecting which journals are likely to provide peer review. And while Bohannon reports that Beall was good at spotting publishers with poor quality control (82% of publishers on his list accepted the manuscript). That means that Beall is falsely accusing nearly one in five as being a ”potential, possible, or probable predatory scholarly open access publisher” on appearances alone. (Reference: Phil Davis: http://scholarlykitchen.sspnet.org/2013/10/04/open-access-sting-reveals-deception-missed-opportunities/)
    Punishment and encouragement are equally important for the development of this new OA publishing industry.

    I have following questions:
    1. Has DOAJ any plan to encourage the small publishers who rejected Bohannon’s article after rigorous peer review?
    2. Has OASPA any plan to encourage the small publishers who rejected Bohannon’s article after rigorous peer review? May be an letter of appreciation to carry out good peer review practice.
    3. Has J Beall any plan to encourage the small publishers who rejected Bohannon’s article after rigorous peer review?

    May be people are afraid to touch so called ‘untouchable predatory publishers’ who are sincerely trying to become good publisher and successfully passed the ‘Bohannon-test’. We are so happy and relaxed that at least big 3 (PLOS, BMC and Hindawi) passed the test. So Open access is saved. After all we are afraid of our social reputation and we can not risk it by touching the ‘untouchables’.

    We must find out way to improve the status of so called ‘predatory publishers’. We must find out and relabel the small publishers who are sincerely trying to improve their practice by learning from the errors of the past. Working with elites is easy. But I believe OA is lacking a Mother Teresa (http://en.wikipedia.org/wiki/Mother_Teresa), who is ready to work with the ‘untouchables’ to clean their would with love, passion and patience. (I believe at least some of the so called predatory publishers now deserve a better treatment. If we can spread ‘the story of their improvement’ may be it can be an effective medicine to heal the remaining disease).

    And what of those journals that have been labeled “predatory” yet rejected this paper? If one mistake is enough to damn a journal, shouldn’t one example of proper behavior be enough to clear a journal’s name? If the point of these registries and blacklists is to improve OA culture and reduce predatory journals, shouldn’t there be a route provided for those who wish to improve their policies and clear their name? Or is being labeled “predatory” a permanent label, from which there is no rescue, and hence no motivation for trying to improve behavior and become a good publisher?

    Akbar Khan
    India

    • Thanks Bill for the reminder. As it is the case of OASPA DOAJ focussed on the journals that accepted the article. We do not have resources to highlight and praise all the publishers who are doing a good job – other than continue to develop DOAJ in such a way that the public can trust, that when a journal is listed in the DOAJ it is a “good” journal. On that note we know that we still have a job to do. Yesterday we launched the new application form for journals to be listed in DOAJ – http://doaj.org/application/new – press release here: https://docs.google.com/document/d/1dr3jnOygvuDlONSBv8lho4McQsEPFd0a5gtxjCmKd9k/edit. The new application form will apply for both new journals to be listed and journals currently listed. So hopefully it will be a quality stamp in itself being listed in DOAJ. It is our mission to help publishers do a better job, and contribute to the transparency and credibility ofopen access publishing.

  2. Thank you for these additional comments. I can only speak on behalf of OASPA and so am not able to reply to the questions regarding the inclusion of these journals on Beall’s list, or any follow-up action that he may have taken. Neither can OASPA represent the DOAJ. Questions raised for other organisations or individuals should be put to them directly.

    OASPA can only react in relation to our own members that have accepted the article and is not in a position to regulate or otherwise intervene with publishers or journals that are not members. At OASPA, an application to join is almost never immediately successful because there are usually issues that a publisher needs to improve on in relation to publication fee information, policy or presentation around licensing, clarity about peer review processes and so on – a considerable amount of work goes into providing guidance and helping smaller publishers or individual journals with these issues. We have also worked recently with DOAJ, WAME and COPE to promote best practices that are relevant to the members of all of these organisations. In these ways we hope to help all publishers – new as well as established – maintain the highest standards, and to ensure that membership is an indication that these standards are being achieved.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>