September 19 to 23 sees the international celebration of Peer Review Week in the academic community, emphasizing the central role peer review plays in scholarly communication. The theme for 2022 is Research Integrity: Creating and Supporting Trust in Research.

What is peer review?

In broad terms, peer review is the pre-publication evaluation of scholarly work by experts in the same field as the submitting authors or with expertise in the methodology they chose. It is common in academic publishing and helps ensure the rigor of publications. Its aim is to either help improve or reject submitted papers that do not meet minimal criteria of good scholarly practice, originality, and methodology. The purpose of this quality control is to build and maintain trust in the published scholarly content, the publishing platforms, and the research process as a whole. Reviewers are invited by the editors or suggested by the authors. Referee activity is considered a courtesy and an academic honor, reflecting a certain reputation and expertise that a reviewer has gained. It is not usually compensated.

A brief history of the peer review process

It is challenging to determine precisely how old the academic peer review process is. While some historians of scholarship have dated it back to the pre-Gutenberg era, and others quote Francis Bacon as one of its trailblazers, many histories of the scholarly system agree that the origins of the contemporary peer review system can be traced back to the editorial practices of the learned societies in the early to mid-18th century, with the Royal Society of London commonly named as one of the main originators.

Contemporary peer review slowly emerged in the second half of the twentieth century. The steadily growing volume of scientific publications called for a screening process, and the newly invented Xerox photocopier made it possible to send out copies of manuscripts to multiple reviewers on a large scale. It was not until the 1960s and 1970s that some of the most reputable journals in academia embraced the practice (Nature in 1964; The Lancet in 1976). Today it is a well-established system, guided by standards and principles that preserve it as one of the pillars of the academic publishing ecosystem.

The increasing awareness of the concept in the broader public is the latest chapter in the history of peer review. Previously it was known primarily to an expert scholarly audience. The Covid-19 pandemic changed that. The tremendous demand for readily available knowledge about the virus led to an unprecedented acceleration of related research. To make the exponentially growing SARS-COV-2 research available as quickly as possible, interest in preprints – scientific papers published before peer review on dedicated servers – increased. Because these preprints are now more commonly used as primary sources, explanations of the peer review process have since found their way into journalistic reporting on scientific topics.

Different types of peer review

Since its establishment, the peer review landscape has diversified. Not only are there different approaches to traditional peer review, but with the advent of the open scholarship movement, newer peer review practices have emerged. They break with some of the established practices of classical peer review, such as anonymity (in Open Peer Review) or confidentiality (in Social Peer Review).

The terminology around peer review is not always used consistently, but some procedures and their terms have become largely accepted. The main types of traditional peer review are commonly distinguished by their approach to anonymity. Anonymity is seen as a critical factor in traditional peer review to eliminate or minimize potential bias among reviewers. Any comments and editing suggestions by the referees remain confidential and are not published along with the work.

• Single-blind PR – Reviewers are aware of authors’ identities.
• Double-blind PR – Neither authors nor referees know each other’s identities.

Recently, the term “blind” has come under criticism for being ableist and a number of journals and publishing platforms have shifted to referring to it as “anonymous peer review”.

Newer, innovative types of peer review step away from anonymity/confidentiality and include:

• Open Peer Review – The identities of authors and reviewers are known to each other and sometimes revealed to the public (there are other interpretations of this term).
• Transparent Peer Review – The identities of authors and reviewers are known to each other, and any comments and editing requests by the referees will be made publicly available. The published article usually has an accessible version history.
• Social or Community Peer Review – The wider (academic) community is invited to participate in reviewing a submitted work and suggest changes. These suggestions and the resulting revisions are usually documented publicly. This approach can be found in the form of pre-publication or post-publication reviews.

Common critique of the peer review system

Critics of traditional peer review question whether it is adequate in a scholarly environment evolving toward more open procedures and principles. Commons criticisms include:

• Delay in publication
• Reviewer’s biases
• Low inter-reviewer reliability across suggested revisions
• Lack of anonymity in highly specialized topics

While many critics believe the peer review system needs improvement and some are calling for its elimination, there seems to be an ongoing consensus among an academic majority that the system is a foundation of academia.

Current discussions

• The paradoxical effects of the publish or perish paradigm on peer review – Many (early career) scholars have no time to devote to the peer review system because they are under growing pressure to publish their own work. As an effect, journals and book presses are drawing on a shrinking pool of reviewers.
• The lack of credit or reward systems for reviewers – Some discussion revolves around most reviewer activity being invisible and if either rewards (e.g., in the form of compensation) or credit (by acknowledging a reviewer’s contribution to a publication, sacrificing the principle of anonymity) should be more prevalent. Some publishers have come up with ways to give credit or compensate reviewers.
• Inability to find factual errors and detect fraud – Numerous studies have shown that peer review is no guarantee against plagiarism, error, or even fraud in scholarly publishing. This is likely to get worse with the fast evolution of writing and paraphrasing tools based on machine learning technology.
• Dubious journals and (the lack of) peer review – The number of fraudulent journals that advertise peer review but do not conduct it is increasing and they are getting harder to identify.
• The direction of the reformation of the peer review system – Some of the current trends and approaches to addressing some of peer review’s issues are pulling in opposing directions.

This is only a selection of current discussions. Peer review is a broad topic that is studied extensively not only in scholarly communication, philosophy of science, and scientometrics but also in individual academic disciplines themselves.

Further Information

To learn more about peer review and Peer Review Week, consult the Peer Review Week committee’s official blog, their Youtube Channel, or Scholarly Kitchen’s series of articles in celebration of the event. On Twitter, follow the handle @PeerRevWeek and use the hashtag #peerreviewweek22.