During a pandemic it can be tempting to publish new research results as quickly as possible. But at the Faculty of Health at Aarhus University, we still advise researchers to allow their new results to be assessed by their peers before they are published. Despite the WHO calling for the opposite.

2021.04.21 | Hans Erik Bøtker, professor, prodekan for forskning ved AU Health

The other day I read an article in a Danish newspaper which, in English, would be entitled "New study: People with glasses have a lower risk of infection with the coronavirus."

The article referred to the results of a research project from abroad, and maybe the conclusion is correct and maybe it is not. The study was not peer-reviewed. 

Peer review is a mechanism for assuring the quality of new results in the world of research. In practice, this means that the research quality is assessed by knowledgeable colleagues before an article can be included for publication in a scientific journal. The assessment is typically anonymous, but it can also be done by a named editorial committee. At least one assessor must be external.

Comments and corrections can be exchanged several times, and this kind of thing can take weeks.

During the coronavirus pandemic, we have therefore seen researchers around the world publish new results online in so-called preprints in order to accelerate the exchange of information. It is perfectly understandable that medical doctors and researchers would want to quickly share new findings with each other and the general public. When the pandemic really took off in 2020, the World Health Organisation (WHO) even backed the trend and encouraged all researchers to publish Covid-19 results before their peers had assessed the content.

But what risks are we taking when we publish new research that avoids the traditional quality check? This is a question which splits researchers around the world into two camps – both with good arguments.

More and more people claim that the peer review process, which became widespread in the 1960s and 70s, is a relic of the past. As early as 2010, an analysis from the French National Institute of Health and Medical Research showed that a total of 93 per cent of the research that was published about the SARS epidemic in Hong Kong and Canada in 2003 was actually first published after the epidemic was over. It goes without saying that this is inopportune.

Critics also point to social media as having long ago democratised the process and made peer review superfluous, as experts around the world can now scrutinise newly published research and point out problematic issues with one click. Neither does it go unnoticed on Twitter when professors with several thousand followers expose their peers’ errors.

On the side of truth

The current system for quality assurance certainly does not guarantee flawless research. Peer reviews do not necessarily detect that the contributor has miscalculated, coloured their formulations or failed to fully maintain their independence from financial backers. The systematic quality check is far from flawless, but the question is: what is the cost of setting it aside?

In 2017, the Ministry of Higher Education and Science published a so-called population barometer, which showed that researchers have a very high level of credibility among the Danish population.

According to the survey, sixty per cent of Danes believed that it is possible to place a high or very high degree of trust in researchers, and that Danes generally have a greater degree of trust in information from university researchers than information from “people like us”, “special interest organisations”, “directors” and “politicians”.

This trust is essential for a well-functioning society. At a time when both there is a greater supply of news and fake news than ever, research remains a beacon in the dark. Research results from independent universities are a vital part of a sensible democratic debate when we discuss major decisions about society. This is reflected when politicians, in connection with the coronavirus pandemic, bring in health experts as a source of truth to give decisions greater legitimacy. In the coronavirus debate, an expert recommendation has been an argument that is almost impossible to trump. The researchers stand on the side of truth, and science contributes with facts that help to qualify the debate – but only as long as we believe in it.

The peer review system is a safeguard against erroneous research findings reaching the general public. As researchers, we cannot withstand making mistakes too often before our credibility suffers – and when you lose credibility, in the long term you also lose the chance to bring new knowledge into the public domain. If mistrust in research increases, it will be detrimental to society.

Unexpected side effects

The stream of preprints about the coronavirus is a challenge, because it can be difficult for people who are not experts - journalists, politicians and researchers in other areas - to determine what is based on solid science and what is not. When things go wrong, the most glaring cases damage public health, because the wrong results are very quickly stored in our collective consciousness. We saw this, for example, with the myth that the MMR vaccine can cause autism. This arose in 1998 through a flawed article in the Lancet by the British medical doctor Andrew Wakefield. The article was withdrawn, but two decades later, the link between the MMR vaccine and autism is still pervasive in some circles, even though it has been repeatedly disproved by large Danish studies, among others.

Despite peer review, surprises may still occur. The current debate on the Covid-19 vaccines from AstraZeneca and Johnson & Johnson are a good example. The clinical studies of the vaccines documented only a few and expected side effects, but distribution among large population groups has led to suspicion of unexpected side effects in the form of blood clots.

Whether the correlation is causal remains uncertain, but rare side effects are not necessarily uncovered in extensive clinical studies, even though the research is peer-reviewed. The health authorities therefore monitor new treatments, and apparently this helps us to maintain our trust in science, even when the results are challenged by the reality. This is supported by a new study from Aarhus University, which shows that nine out of ten Danes will get the Covid-19 vaccine when it is offered to them. This is the same level as before the AstraZeneca vaccine was suspended.

Things can always be improved

The peer review system is not perfect, but my experience tells me that it raises the level of research. I have been a supervisor for many PhD students, and I have always advised them to send an article to peer review before submitting their dissertation. In the vast majority of cases, the research is improved by feedback. Both as a researcher and supervisor, you can overlook things, look too hard at something, and unconsciously influence one another in a particular direction. Therefore, it is important to have an external expert’s eye for detail, and it is very rare for peers not to have suggestions for improvements. Throughout a long research career, I have only once experienced that there were no comments in the peer review process.

There are also ongoing improvements of science's own quality assurance system – for example, some of the traditional journals have made accelerated peer review processes possible during the coronavirus pandemic, so that the assessors do not have 14 days, but only 48 hours to assess the publication.

Researchers are, of course, allowed to do as they please – and there may be grey areas where special considerations apply – but we must remember to discuss how we ultimately serve science and society best. Peer review reduces the risk of misinformation and supports the credibility of the researchers and science. At the Faculty of Health at Aarhus University, we strive to ensure the highest standard of research in relation to what is possible. Therefore, we encourage researchers to sacrifice the time it takes to get a qualified peer review.

The text was published as a column (in Danish) in Jyllands-Posten on Tuesday 20th of April.