How to Fix Peer Review and Cure Science's Growing 'Cost Disease'

Story Stream
recent articles

The engine of the modern scientific enterprise is peer review. When scientists complete a new study and seek to get it published in an academic journal, the journal's editor will send the paper out to some of the submitting scientist's colleagues to critique it. They'll judge the paper's quality and provide learned feedback. Remarkably, they do this for free, despite the fact that academic publishers are generally quite profitable, making money by disseminating finished scientific papers to libraries, academic institutions, and other researchers.

Last year, psychologists Balazs Aczel and Barnabas Szaszi, of Eotvos Lorand Univesity in Hungary, and Alex O. Holcombe, at the University of Sydney, calculated researchers’ time and salary-based monetary contribution to peer review in 2019, and made their findings available free online. By estimating the total number of reviews, based on the average reviews for the more than 7 million accepted and rejected scientific articles in 2019, then multiplying that number by the average time spent per review (about six hours according to a 2009 survey of 3,597 reviewers), the trio estimated that in 2019 researchers spent 108,492,744 hours on reviewing, equivalent to 12,385 years!

Again, this is unpaid labor.

If it wasn't free, and reviewers (mostly post-docs and professors) were compensated for their time based on their average salaries, then reviewers in the United Kingdom would have earned $266 million in 2019, according to the authors. In China, that tally would be $665 million. In the United States, it would be $1.1 billion!

So what is the solution to this blatantly unfair status quo?

An obvious answer is for publishing companies to compensate researchers for providing peer review. Most scientific journals, which are in turn owned by large publishers, should be able to handle the expense. Elsevier, which publishes 18% of the world's scientific articles, earned $1.3 billion in operating income on revenue of $3.6 billion in 2019, a profit margin that rivals Apple's. Assuming that there's another $1 billion worth of peer review conducted globally, that would place the total worth of peer review on Earth at about $3 billion. If Elsevier had to pay about a fifth of that to compensate reviewers (roughly equal to their 18% market share), the company would still make $700 million in operating income each year. Clearly, Elsevier can afford to pay reviewers.

In addition to compensation, Aczel, Szaszi, and Holcombe see three other ways to streamline peer review. "One is to decrease the amount of labor needed per published article by reducing redundancy in reviews," they wrote. "A second way is by better allocation of review labor. A third way is by unlocking the value of peer reviews, in part by making them publicly available."

They note that many manuscripts are often submitted to and reviewed by multiple journals, a redundant process. If reviews conducted at one journal for a certain paper were made available to another journal, reviewing time could be substantially reduced. They also suggest that peer review could be transformed from a relatively secretive to a more open process by soliciting reviews for papers on online forums or marketplaces accessible to researchers. This way, researchers could potentially "bid" for the opportunity to review papers. Lastly, maybe even the reviews, themselves, could be freely available.

"Because peer reviews are traditionally available only to the editor and authors of the reviewed study, this dissemination of knowledge happens only very slowly, much like the traditional apprenticeship system required for professions before the invention of the printing press. How much more productive would the scientific enterprise be if the information in peer reviews were unlocked?" Aczel, Szaszi, and Holcombe wrote.

The trio ended their paper with a warning.

"Unless the efficiency of peer review is improved, then science faces a cost disease situation. “Cost disease” refers to the fact that while many products and services have steadily decreased over the last two hundred years, those that have remained very labor-intensive have not. This is the fate of scholarly publication, unless something is done to make peer review more efficient."

Show comments Hide Comments