The British magazine The Spectator referred to research published in Norway to back up the magazine’s claim that societal lockdowns are not an effective means to reduce the spread of COVID-19 infections. This unlikely occurrence highlights some of the difficulties in ensuring that research has a societal impact.
Norwegian shop in March 2020. Shelves are emptied by people stocking up for insecurity. Photo: Wikimedia Commons CC 4.0
Funders and taxpayers reasonably expect that research should be relevant and contribute to socially useful outcomes.
One way for researchers to meet these expectations is for their work to be used to inform public debates about government policy. It seems preferable to have such debates based upon scientific evidence rather than common sense or ideology. However, the route from a research publication to discussion in the media can be convoluted and impossible to predict.
Along the way, meaning may be changed as important qualifications and caveats are removed. The impression about the research presented in the media may be different to what the authors originally intended. All of these complications are intensified when the research is being used in highly politicized debates – in this case over government restrictions on personal liberty as a response to a pandemic.
The lockdown wasn’t needed in Norway, apparently
On 27 May 2020 Fraser Nelson, editor of the British current affairs magazine The Spectator, wrote an article about a report on Norway’s response to the COVID-19 epidemic that had been published three weeks previously. The article, entitled “Norway health chief: lockdown was not needed to tame Covid”, is used by Nelson to support The Spectator’s call for the end of UK legal restrictions aimed at controlling the epidemic, the magazine emphatically argues that “It is time to restore liberty and move to a voluntary system”.
Nelson’s article raises some important questions and dilemmas for researchers and how they achieve societal impact.
The article in The Spectator states that:
the Norwegian public health authority has published a report with a striking conclusion: the virus was never spreading as fast as had been feared and was already on the way out when lockdown was ordered. ‘It looks as if the effective reproduction rate had already dropped to around 1.1 when the most comprehensive measures were implemented on 12 March, and that there would not be much to push it down below 1… We have seen in retrospect that the infection was on its way down.’
The original sources
The original report, published by the Norwegian Institute of Public Health (NIPH), comes to a different conclusion.
A paragraph on page 24 is as quoted above, and ends with the words “down below 1”. However, the next paragraph has a much less emphatic tone than recounted by Fraser Nelson. The NIH report states:
From this it follows that we don’t know what measures have to be kept in place to keep the epidemic at a level where the capacity of the health service is not exceeded and negative repercussions are not too great.
Av dette følger at vi heller ikke vet hvilke tiltak som må opprettholdes for å holde epidemien på et nivå der helsetjenestens kapasitet ikke overskrides, og der de negative ringvirkningene ikke blir for store.
This is an important qualification about uncertainty which was included by the researchers who wrote the original report, and which was not present in The Spectator article.
Camilla Stoltenberg, head of the Norwegian Institute of Public Health. Photo: NordForsk / Terje Heiestad / Wikimedia Commons CC 2.0
But what about the much more certain sentence that “We have seen in retrospect that the infection was on its way down”? That statement didn’t come from the NIPH report. Instead, it comes from an article by Norwegian public service broadcaster NRK about an interview on 21 May with Camilla Stoltenberg, director of the NIPH, in which she is reported as having stated:
At some point it may be summarized, if it is possible at all to summarize with a high degree of certainty. With hindsight we have seen that the infection was on its way down, it has taken time to obtain the data needed for analysis.
Det kan hende at det blir oppsummeringen, hvis man i det hele tatt greier å oppsummere med en stor grad av sikkerhet. Vi har sett i etterkant at smitten var på vei ned, det har det tatt tid å få data til å analysere.
Tracing the likely route
It is unlikely that Fraser Nelson monitored the Norwegian media himself. Instead, it appears that there was a somewhat convoluted path from the NPIH report to The Spectator. A crucial step was that on 22 May The Local, which publishes Norway’s news in English, published a summary of the interview with Stoltenberg. Links to that summary were shared on Twitter by ‘lockdown skeptics’ and the story was also written up by The Washington Examiner a day before the Spectator article.
Curiously, the phrasing in the article in The Local is somewhat different to that which appeared in The Spectator. The likely path was revealed with my realization that the output from Google Translate of the NRK article is much more similar.
In particular, the NRK article segues from a quote from the NIPH report about the reproduction rate to remarks by Stoltenberg which include the ‘in retrospect’ comment. It is possible that Nelson checked the NRK article with Google Translate, but accidentally elided the two quoted statements.
Researchers are encouraged to attain societal impact, which is often measured by the extent to which their work is used by decision-makers or cited in public debates. This short case illustrates some of the complexities and difficulties involved.
Societal impact is complicated
Firstly, the NIPH report received a lot of exposure from The Spectator article. Fraser Nelson has over 242 000 followers on Twitter, and his tweet announcing the article “Norway says if [sic] has found that Covid was under control before it locked down – it now accepts that the virus could have been dealt with using other means.” has received over 700 retweets, and over a thousand likes. In contrast, the NIPH tweet announcing the original report received 4 retweets and 13 likes. In terms of crude altimetrics, via Fraser Nelson the NIPH hit the jackpot.
Secondly, a convoluted process removed the careful qualifications contained in the report and in the comments by Stoltenberg. That impact bonanza was achieved at the expense of the meaning that was intended by the original authors of the report. When assessing societal impact it is important to understand not just how often research is used or cited, but the extent to which meaning has changed along the way, and whether any such content drift is detrimental to the findings presented by the original authors.
Thirdly, the path from publication of the report to societal impact would have been impossible to predict beforehand.
The conclusions of the original, and to English speakers what would be an obscure publication, were likely mediated via a public servant (Stoltenberg) being interviewed by a broadcaster, translation into English by The Local, diffusion via social media, translation again via Google, and publication by The Spectator. This path shows that research can play a role in public debate even if the original authors played no active role in the dissemination process.
Finally, the NIPH report achieved such societal impact because the lockdowns in the United Kingdom (and United States) have been highly politicized.
In such an environment the experience of countries such as Norway are used by many different actors to advance their agendas. Sweden has become, to a much greater extent than Norway, the subject of highly politicized debate far beyond its borders.
Such contentious environments are well known to social scientists working in fields such as economic policy, crime or migration, and are similarly familiar to natural scientists working on subjects such as climate change. This example shows that the social environment in which research takes place can change rapidly, and subjects like virology or epidemiology can suddenly become highly political arenas.