If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Heart Lung and Circulation, Sydney, NSW, AustraliaDepartment of Cardiology, Westmead Hospital, and University of Sydney, NSW, AustraliaDepartment of Cardiology, Blacktown Hospital, and Western Sydney University, NSW, Australia
Embarking on conducting peer reviews for academic journals can present a new and exciting challenge for early career researchers. This article offers succinct guidance about peer review: not only “what to do” (the Good) but also “what not to do” (the Bad) and “what to never do” (the Ugly). It outlines models of peer review and provides an overview of types of reviewer bias, including conflict of interest. More recent developments in journal peer review, such as author-suggested reviewers as well as manipulation of the peer review process are also discussed. A new position of Editorial Fellow at Heart, Lung and Circulation will provide aspiring researchers the opportunity for multi-faceted involvement with peer review at the Journal.
], followed by a series of support sessions for aspiring authors held as satellite events at Annual Scientific Meetings of the Cardiac Society of Australia and New Zealand. Commencing in 2016 with “What Editors are Looking for” [
], the 2018 session expanded on peer review in “Everything You Need to Know about Peer Review.” This article, based on that session, presents an overview of the processes and practices — good, bad, and terrible — in peer review for academic journals. Peer review of research grant proposals was beyond the scope of this presentation and is not discussed.
What is Peer Review?
At its simplest, journal peer review has been described as “the assessment by an expert, of material submitted for publication” [
]. Linked to the development of the scientific method as described by Francis Bacon in Novum Organum in 1620, peer review was instigated in the mid-18th century by Royal Societies in the UK, with the Philosophical Transactions of the Royal Society of London generally acknowledged to be the first journal to use a formal peer review process [
]. A slow and gradual adoption in peer review proceeded, which sped up alongside technological advances in the 20th century. (i.e., the typewriter, the photocopier, the fax, and finally, the internet), as well as increasing specialisation. In the 1960s, Ziman went so far as to say peer review was the “the lynchpin about which the whole business of Science is pivoted” [
Probably since its adoption, peer review has been subjected to much ongoing criticism related to perceived non-objectivity or bias, non-transparency and possible lack of rigour. In 2002, Sir Peter Lachmann, then President of the Academy of Medical Sciences opined that “Peer Review is to science what democracy is to politics. It’s not the most efficient mechanism, but it’s the least corruptible” [
]. Despite its limitations, peer review remains the main way that the quality of submitted manuscripts is ensured prior to acceptance for publication.
A typical journal peer review flow diagram is shown as Figure 1. Importantly, submitted manuscripts can be rejected by journal editors on initial screening, that is, without peer review. Thus, generally only manuscripts thought to be appropriate and of sufficient merit will progress to peer review, with the advice received assisting the editor to decide whether the manuscript will proceed further towards publication.
Articles with a “Similarity Index” higher than a threshold such as 5% will also be flagged to the handling editors.
Why Does Peer Review Matter?
All academic peer-reviewed journals are critically dependent on high-quality peer review, and we consider the purpose of peer review is to filter what work will find its way into the literature. This process allows journals like Heart, Lung and Circulation to reassure readers that only articles that are both credible and relevant are being published. The peer review process generally results in articles appearing in the Journal in better condition than when they were first submitted; apart from improvements in clarity of writing and enhanced presentation, relevant literature and limitations of methodology may be better acknowledged, and over-reaching conclusions moderated [
Most importantly, we would agree that such work is more likely to lead to patient benefit rather than the potentially “frightening downstream consequences” of work published by other, superficially or even “cursorily” reviewed journals [
In brief, the main models of peer review are based on identifiability of the peer review process:
•
Single-blind, where the authors are known to the reviewer but not vice-versa;
•
Double-blind, where neither authors nor reviewers are identifiable to each other; and,
•
Open review, where both authors and reviewers are known to each other.
Other variables in peer review relate to the timing review – pre-print, pre-publication and post-publication, and the publication of reviews – not published, published anonymously, or “signed”.
Each of the three main models has its supporters and detractors. For example, it is thought anonymous reviewers (in single-blind and double-blind review) may be more direct in their opinions; that open review is subject to public scrutiny; and that double-blind review, while providing some protection from reviewer bias related to author characteristics, may reduce awareness of potential conflicts of interest.
Reviewer Bias
Reviewer bias is one of the key common criticisms of the peer review process. It has been defined as “a systematic prejudice that prevents the accurate and objective interpretation of scientific studies” [
Bias may relate not only to author characteristics such as geography, nationality, language, specialty, gender and affiliation or prestige but also reviewer characteristics, such as preferences for type of content (e.g., by topic), type of study (e.g., bias against observational work), bias for or against interdisciplinary research, confirmation bias (i.e., tendency to endorse work in line with one’s own beliefs) and publication bias (i.e., a well-documented trend for trials with negative results to not be published and, correspondingly, for trials with positive results to be published).
Reviewer bias may not be easily detectable; and, it may not be correctable.
Conflict of Interest
Conflict of interest has received much attention in recent years, especially with respect to authors’ potential competing interests; however, it is equally relevant as a type of reviewer bias.
The World Association of Medical Editors (WAME) has defined conflict of interest as follows:
“Conflict of interest exists when there is a divergence between an individual’s private interests (competing interest) and his or her responsibility to scientific and publication activities such that a reasonable observer might wonder if the individual’s behavior or judgement was motivated by consideration of his or her competing interest.”
Accordingly, conflict of interest can be personal, financial, academic, competitive and/or intellectual.
Depending on the conflict, a reviewer may still be able to proceed to peer review the manuscript; however, the reviewer should first let the editor know of the potential conflict. The editor will decide whether and how the conflict can be managed, as in some areas of controversy, there can be literal truth in the saying “No conflict, no interest.”
How to Peer Review?
When invited to review a manuscript for a journal like Heart, Lung and Circulation (which conducts pre-publication, confidential double-blind review) we suggest reviewers keep three key points in mind:
•
Are you the right person to do the review now?
Before you accept the invitation to review, it is useful to consider whether you have the relevant expertise, no (or manageable) conflict of interest, and enough time to complete the task in a timely manner.
•
Keep it confidential:
Before you write up your review, think about what you have read but don’t discuss the paper with another colleague without asking the editors first.
•
Follow a Format:
When you do write up your review, be professional, succinct and direct, and follow a format that is either provided by the Journal or that works best for you.
Based on our collective decades of experience as Journal editors, we offer the follow further advice about: what to do (The Good); what not to do (The Bad), and, what to definitely avoid (The Ugly) in the practice of Peer Review.
The Good
The best reviews are objective, constructive and strike an artful balance between being comprehensive and succinct. Good reviewers will generally be systematic, specific and “nice” (i.e., polite and professional) in their approach to reviewing, and follow a clear format. Good reviewers are likely to:
•
Summarise the work under review, address its strengths, then main weaknesses, and specifically identify any “deal-breakers”, such as possible plagiarism, that could preclude publication.
•
Provide specific comments on design, presentation of data, the results and the discussion.
•
Specifically comment on the originality, and the significance or clinical and/or policy implications of findings.
•
Share concerns, such as any important omissions (e.g., key references) or potential conflicts of interest.
A good review will also include a clear Recommendation to the Editor — whether it is to Accept, seek Minor or Major Revision, or to Reject the manuscript. Lastly, a Good Reviewer will submit their report on time, and indicate whether they would be willing and available to review a revised version of the submission.
Guides to Good Reviewing are widely available, both generally (Box 1) and specifically, for individual journals, and for specific types of submissions. For example, Moher offered optimal strategies to consider when peer reviewing a systematic review and meta-analysis, recommending the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement [
]. Del Mar and Hoffman offered guidance in performing a peer review of randomised controlled trials, again promoting awareness of appropriate study reporting checklists, such as the CONsolidated Standards Of Reporting Trials (CONSORT) Statement [
]. Byrne has proposed a summary of issues for peer reviewers to consider when reviewing narrative literature reviews and assessing their significance, included was the conduct of literature searches defined, were original references cited, was information summarised correctly, were studies critically evaluated, and does the review expand the existing body of knowledge [
COPE (Committee on Publication Ethics) Council. Ethical Guidelines for Peer Reviewers. September 2017. www.publicationethics.org
•
ICMJE (international Committee of Medical Journal Editors) Recommendations for the Conduct, Reporting, Editing and Publication of Scholarly Work in Medical Journals. www.icmje.org/recommendations/
Importantly, the peer review process is intended to be confidential and is expected to remain so, and a good reviewer will always honour this ethical principal, as it is the foundation of good peer review.
The Bad
A thoroughly “bad” review could be considered the (unsurprising) corollary of a good one. However, elements of a bad review are not uncommonly found in otherwise exemplary reviews. Elements of less-than-optimal reviewing might include:
•
providing general, non-specific comment about a manuscript, without supporting evidence.
•
including a recommendation to extend the reported work well beyond its current scope as a condition of publication
•
seeking self-citation of a reviewer’s own work when not relevant to the topic under discussion, and
•
breaking confidentiality or, without first consulting the Editor, involving others in the process of review.
One of the more frequently encountered elements, which can be frustrating for Journal editors, is when a reviewer provides a Recommendation to the Editor inconsistent with their Comments to the Author. For example, when the “Recommendation to the Editor” is to reject the manuscript or seek a major revision yet “Comments For The Author” are complementary or minor in terms of requested amendments.
And The Ugly
The extreme, and hopefully rare, occurrence of an “ugly” reviewing is likely to involve academic misconduct. As the COPE Guidelines clearly state, reviewers should not be agreeing to review a manuscript just to gain sight of it, with no intention of submitting a review; nor should reviewers agree to review a manuscript very similar to anything he or she may be preparing to submit or have already submitted elsewhere [
]. There are anecdotal reports of reviewers who have “sat” on manuscripts so that they may submit their own similar paper first elsewhere. Even worse, van Rooyen et al. said reports exist that anonymous reviewers have appropriated ideas from manuscripts they have reviewed [
Confidential comments to the editor should not be a place for denigrating others or making accusations, offered in the knowledge that the author/s will not (or are unlikely to) see such comments [
]. We suggest that reviewers take the view that - like for emails - anything written to any other person can, at some point, be seen be others. At all times, we would recommend a polite and professional approach.
Even uglier abuse of peer review involves “predatory reviewers” who steal or plagiarise other authors’ ideas [
COPE has advised editors of features or patterns of reviewer activity that can suggest manipulation of peer review, particularly when several of these are evident. We have shown some of these in Box 2.
]. There have now been quite a few reported examples of such “fake” reviews, which when detected have led, among other things, to article retraction and professional repercussions [
]. Generally, the authors were exploiting a feature of a manuscript tracking system’s automated processes: when asked to suggest reviewers, they provided the names of real or fictitious people, with bogus email addresses, that would then be directed to the authors or their associates [
]; in one case, a third party (a company) was involved in the submission process that offered biographies and email addresses of experts which appeared to be fabricated [
As the number of journals available for publication rises, straining the resources of peer reviewers, alternative models of peer review and publication are being explored, some more controversial than others.
“Cascading” or waterfall peer review, also known as transferring or portable review, is available with some of the larger publishers, where authors’ submissions (and peer reviews) may be redirected after rejection from the initial journal of submission to others in the publishing group's “stable”, with movement generally occurring “downwards”, from the more highly regarded, higher impact journals to less prominent titles.
Diversity and inclusion in peer review has also received attention, for example, as the theme of the international Peer Review Week in September 2018, which aimed to raise awareness of issues surrounding gender, underrepresented minorities and country bias, as well as the need to include early-career researchers in peer review.
There is also movement towards making peer review more open, in line with the belief, as van Rooyen et al. stated, that science needs to catch up with the rest of the world, where transparency of decision making is becoming the norm [
]; that while peer review does produce quality judgments, one does not quite know how they come about. Further, while there have been some randomised controlled trials of peer review interventions [
], this cornerstone of scientific advance remained largely under-researched, it is said, principally because of the “hidden” status and issues of confidentiality [
These concerns have no doubt also played a role in several movements towards open peer review, where both the reviewers’ comment and identity may be made publicly available, and an interest in post-publication review, where open access manuscripts may not be critically reviewed until they are in the public domain.
More recently, the EU funded New Frontiers of Peer Review (www.peere.org/peeer-in-a-nutshell), which aimed to improve the efficiency, transparency and accountability of peer review through trans-disciplinary collaboration and evidence-based peer review.
Author-Suggested Reviewers
Some journals, including Heart, Lung and Circulation, ask or allow authors to suggest or list peer reviewers as “preferred”. Authors may also wish to list “opposed” reviewers. In Heart, Lung and Circulation’s case, these author suggestions are sought as a possible source of credible opinion when the Editors’ own known or preferred reviewers are unavailable or not interested in reviewing the manuscript. Although a pragmatic reaction to a current and ongoing dearth of available reviewers, Hausmann et al. recently reported a bias toward lower rejection rates when author-suggested reviewers assessed papers submitted to the Journal of Neurochemistry [
]. Their findings led to that journal’s decision to abandon the option to recommend reviewers. Charlier et al. believe the risk of evaluation bias when authors propose reviewers is immense, with the possibility of authors suggesting friends and colleagues as reviewers [
]. They are also concerned about the room for abuse and the potential for deliberate manipulation of the peer review process, as discussed above.
Why Do Reviewers Review?
Reviewers may opt to peer review for any of a number of reasons: to keep abreast of the latest developments, to assist in developing their own research and writing, to build relationships with journals and editors, to develop their careers and to fulfill a sense of professional responsibility or academic duty.
Formal professional or public recognition of the role and skill of peer reviewers is now receiving more attention. Journals, including Heart, Lung and Circulation, may publish lists of active reviewers (e.g. by calendar year), and invite particularly active and efficient reviewers to join their Editorial Boards.
We believe that peer review at its best involves research community participation, fairness and impartiality, accountability and timeliness, transparency and independence, confidentiality and trust, appropriateness and balance, and continuous improvement.
Heart, Lung and Circulation has always welcomed new peer reviewers, and encourages any reader who has published any relevant research to consider participating in peer review for our Journal.
The Heart, Lung and Circulation Editorial Fellow
We also believe that peer review is an integral part of the academic process and would like to provide an opportunity for up-and-coming researchers to fully experience Academic Peer-Reviewer Publication via a newly created honorary position of Editorial Fellow. The Heart, Lung and Circulation Editorial Fellow will:
•
be a recognised member of the Journal’s Section Editors’ Board, attending Editorial Board and other relevant meetings (e.g., the Annual Scientific Meeting of the Cardiac Society of Australia and New Zealand);
•
receive training in peer review across a range of submissions; and,
•
assist with a Special Issue or Journal events and/or co-author an editorial or other article for publication in Heart, Lung and Circulation.
Eligible candidates for the position of the Editorial Fellow will be based in Australasia, at the advanced trainee or post-doctoral level, with some previous academic authorship experience necessary and preferably some prior experience in conducting peer review.
Formal expressions of interest for the position will be sought from mid-2019, and we look forward to formally announcing the position of inaugural Heart, Lung and Circulation Editorial Fellow at the Cardiac Society of Australia and New Zealand (CSANZ) Annual Scientific Meeting in August 2019.
As well as inducting the position of Editorial Fellow at Heart, Lung and Circulation, we look forward to encouraging early career researchers to join the peer review fraternity and trust they will gain as much from the process as they contribute. We also thank all peer reviewers who review for our and any other journals. It is your contribution which enables the ongoing advance of scientific medical knowledge.
References
Gregory A.T.
Denniss A.R.
Heart, Lung and Circulation evolves: a fond farewell to our 25th anniversary year and a warm welcome to new initiatives.
☆This Editorial is based on “Everything You Need to Know about Peer Review”, a Satellite Event Presentation by Dr Ann T. Gregory and Professor A. Robert Denniss at the CSANZ Annual Scientific Meeting, Brisbane, Qld, Australia, August 2018.
30555-4&id=gr1.jpg){kind=link}