Science journalism can be a minefield of inaccuracy in the mainstream media as science and technology editor, Rob Lea, discovered examining a report regarding ‘Killer bacon’.
As a science journalist and writer, there is very little more irritating than reading a terrible piece of science journalism. This irritation is compounded when it appears that an organisation has taken a piece of research and ‘sexed it up’ to the point at which it barely reflects the research it concerns.
That was the case when I came online this morning and viewed this headline from LBC — a London-based talk-radio station founded in 1973 — that definitely strikes a cord of ‘fear-mongering’ at first glance. ‘Killer Full English: Bacon Ups Cancer Risk’ (1).
Delving into the short article reveals a minefield of mistakes that journalists need to DESPERATELY avoid when reporting the results of a scientific study. It should be noted, I’m not criticising the work of the researchers in question here. The study might be great. It might be poor. What I want to focus on is how the press has reported those findings.
Nor am I saying that the link between colorectal cancer and processed meats doesn’t exist. I certainly not encouraging people not to limit red and processed meats in their diet. Stick to government guidelines–less if you can–there IS good evidence linking processed meats to cancer.
Despite this, reports discussing this relationship need to be properly balanced and accurate. and reporters owe it to their audience to attempt to improve their understanding if they are going to report on the findings of studies.
Also, it should be noted that LBC was hardly the only news organisation to report this study in this way. They were the first I saw and frankly, capped off their article with the most inflammatory headline.
Ah, that headline…
Like any good journo, what I need to do is front load this story with the juice. First off, that headline is deeply misrepresentative of the subject at hand and hyperbolic to a level you rarely see in the news media. You simply can’t just imply that a foodstuff is ‘killer’ without some pretty conclusive evidence.
The headline also draws a causal link between bacon — and by extension other cured meats — and cancers. This link isn’t established in the research paper for a very good reason —one that is in no way a fault of the researchers — the study in question just isn’t fit for that purpose. This is something that LBC has a responsibility to report — one they’ve failed to act on.
Couple this with the fact that there are some extremely misleading statements in the article.
LBC doesn’t understand how WHO classify carcinogens
Take this for example. LBC says:
“ The World Health Organisation have said that processed meat ranks alongside smoking as major cause of cancer, while red meat has been classified as a ‘probable’ cause of cancer.
“The World Health Organisation (WHO) have said that processed meat ranks alongside smoking as major cause of cancer.”
WHO has indeed ranked processed meat in IARC Group 1 —the category used when there is sufficient evidence of carcinogenicity in humans — with tobacco, but that doesn’t mean that eating processed meat is as likely to cause cancer as exposure to tobacco or asbestos (also a category 1 substance).
But WHO actually says (2):
“processed meat has been classified in the same category as causes of cancer such as tobacco smoking and asbestos (IARC Group 1, carcinogenic to humans), but this does NOT mean that they are all equally dangerous.
“The IARC classifications describe the strength of the scientific evidence about an agent being a cause of cancer, rather than assessing the level of risk.”
So the similarity in categorisation says nothing about both factors being “major” causes — the group is based upon likelihood, not risk. This is a major and fundamental misunderstanding that is being communicated to the public.
Red meat has been classified as IARC Group 2A — factors that probably cause cancer — this means there isn’t enough evidence to put red meats into Group 1.
WHO notes with regards to red meats:
“ Limited evidence means that a positive association has been observed between exposure to the agent and cancer but that other explanations for the observations (technically termed chance, bias, or confounding) could not be ruled out.”
Group 2A materials are normally placed there because there is little evidence of correlation in humans, but animal trials have shown experimental evidence.
LBC doesn’t understand why you cite the original research
The study at the centre of this report is ‘Diet and colorectal cancer in UK Biobank: a prospective study’ published in the International Journal of Epidemiology (3). You could be forgiven for not knowing that from the LBC report because they simply don’t mention the title of the paper, nor do they link to it or make the slightest mention of the authors.
They also don’t reach out to the researchers for comment. Something that is absolutely unforgivable in science journalism. In the past, I’ve dropped decent stories because I couldn’t get a comment from anyone on the research team.
What is also missing is a second opinion from a scientist working in this area but not directly involved in the research. This generally serves to add context so the reader can quickly tell if the findings being shared are as significant as they seem.
Had these things been present in the article it might have at least allowed readers to establish the nature of the study properly. I’m aware that few readers are actually going to go searching for peer-reviewed papers based, but they should have that option.
It took me only a short time to find the paper, mainly because the title very closely reflects the conclusions discussed in the article. This isn’t always the case. The titles of research papers can often be deeply obscure unless you’re well-versed in the science being discussed.
couple this with the fact that hundreds of research papers are published That means finding one, as a layman, can be extremely difficult.
That’s why citations are vital. If you don’t include them, then you are not being transparent with your journalism. People may suspect that you are either trying to obscure the original findings or you simply didn’t read them for yourself.
That’s fine. Let’s be realistic, most journalists don’t have time to read every paper they report on. That’s why we contact the authors and other experts in the field.
LBC doesn’t understand the limits of epidemiological studies
The study in question at the centre of this article is an epidemiological study. LBC doesn’t mention this beyond the use of the word in the title of the journal the paper found itself published in. This may well be because they don’t understand the importance of pointing this out.
Epidemiological studies are generally the type of research we do when we research disease in humans. As such, we can’t cast aspersions on the usefulness of this kind of study. They are responsible for a lot of the knowledge we have regarding disease in humans.
But it is vital to note that epidemiological studies have severe limitations.
The American Cancer Society says (4):
“Humans do not live in a controlled environment. People are exposed to many substances at any given time, including those they encounter at work, school, or home; in the food, they eat; and in the air, they breathe. It’s very unlikely they know exactly what they’ve been exposed to or that they would be able to remember all of their exposures if asked by a researcher. And there are usually many years (often decades) between exposure to a carcinogen and the development of cancer.
“Therefore, it can be very hard to definitely link any particular exposure to cancer.”
This type of study is limited because unlike lab studies researchers don’t place any controls on the subjects of the analysis. This means that epidemiological studies don’t attempt to isolate one particular cause of cancer. They are confounded by nature.
Therefore, it is impossible to draw causal links form epidemiological studies. An epidemiological study cannot say “X causes cancer” or even “X increases the risk of developing cancer.”
When it comes to the study in question, the presence of these confounding elements — things like age, smoking, alcohol intake and BMI — seems to have had a significant effect on the findings.
From the study:
“Compared with those in the lowest category, participants in the highest category of reported total red-meat intake were slightly older, more likely to be smokers, had a higher BMI and body-fat percentage, had a higher alcohol intake and had lower intakes of fruit, vegetables and fibre.”
From this information, it seems that those more inclined to eat more red meat were also likely to indulge in several other activities associated with elevated cancer-risk and to have generally less healthy lifestyles.
LBC should point this out. The fact that they don’t means this study is fundamentally misrepresented. Take a look at that headline again, that certainly implies a strong causal link to me.
LBC doesn’t understand the numbers
LBC report that:
“ experts examined data from 475,581 people aged 40 to 69 at the start of the study and followed them for an average of 5.7 years.”
Now 475,000 people is a really respectable sample size, there is just one problem. This was the group that provided initial dietary data via a questionnaire.
The researchers only continued to collect data from ~175,000 — still an impressive sample size, but it’s important to note that the researchers had no idea what happened in the diets of ~300,000 of their initial sample size in the course of that average 5.7 years.
What is more important though is the absence of any absolute numbers. LBC report that eating “ most processed meat” was associated with a 17% higher risk of colorectal cancer. Obviously, the phrase “most” is disconcertingly vague.
Secondly, whilst a 17% risk increase may sound extreme, it only becomes so in light of what the initial risk is. The absolute numbers for colorectal cancer diagnosis in the UK are quite high. 1/15 for men, 1/18 for women.
So a diet high in processed meat, according to this study would increase the risk in men from 20 in 300 to 23 in 300. Not a staggering increase in risk but one that definitely should be considered with regards to diet.
In terms of risk factors though, it is nowhere near the risk associated with a family history of bowel cancer. In the meta-analysis — a study which collects and examines the results of many separate pieces of research — ‘Relative and absolute risk of colorectal cancer for individuals with a family history: a meta-analysis.’(5) researchers found an almost 56% risk increase for individuals with two family members diagnosed. For a person with one family member diagnosed with bowel cancer, the risk increases by 53%.
LBC needs to understand why this matters.
The problem with articles like LBC’s is that they sour the public’s perception of science and scientists in general. They make them less inclined to listen to sound advice.
I have to wonder how many people’s first reaction to seeing this story was “Oh, they say everything causes cancer these days.” These people probably think that it’s scientists at fault here, but it’s actually the news media failing to correctly represent findings.
Possibly because restrained and responsible reporting doesn’t generate ‘hits’ or sell papers. As the old adage goes ‘if it bleeds, it leads’ but that attitude is damaging to science. Ingendering this cynicism towards science and health advice will hurt people, it probably has.
Worse, it could kill people.
We live in a crucial time for human development. As a species, we have become increasingly reliant on the results of science. Yet so few of us are scientifically literate. This means that the media has a responsibility to accurately report scientific findings without sensationalism.
(1) Killer Full English: Bacon Ups Cancer Risk, No byline attribution, LBC, 17/04/19 https://www.lbc.co.uk/news/killer-full-english-bacon-ups-cancer-risk/?fbclid=IwAR0tJxs2g5T8brLh6cZZXgjFjOPZcAvK6Cq-BdksiU2WOtkcsRFL2jO1VpQ
(2) Q&A on the carcinogenicity of the consumption of red meat and processed meat, WHO, 2015, https://www.who.int/features/qa/cancer-red-meat/en/
(3) ‘Diet and colorectal cancer in UK Biobank: a prospective study’, Bradbury. Kathryn, Murphy. Neil, Key. Timothy.J, International Journal of Epidemiology, 2019, DOI: https://doi.org/10.1093/ije/dyz064
(4) Known and Probable Human Carcinogens, https://www.cancer.org/cancer/cancer-causes/general-info/known-and-probable-human-carcinogens.html
(5) Relative and absolute risk of colorectal cancer for individuals with a family history: a meta-analysis, Butterworth. AS, Higgins. JP, Pharoah.P, NCBI, 2006, DOI: https://doi.org/10.1016/j.ejca.2005.09.023