Breast screening: case control vs RCT – the problems

The Daily Telegraph reported yesterday that “Breast cancer screening ‘works and we should move on” – “Women should undergo breast cancer screening because it halves the chance of them dying of the disease, according to a new study that claims to draw a line under the controversy”.

So simple? No!

Here’s the paper. It’s a case control study. So you look back at women who have died of breast cancer, and take a comparable group of women, but who didn’t die of breast cancer. Then you find out if one group went for breast screening more than the other. The researchers found that there was  “an average 49% reduction in breast cancer mortality for women who are screened.”

I don’t think this means that we have proven whether breast screening works or not. The best way to do that is through a randomised controlled trial. In this type of trial, a comparable group of women are sorted into two random groups, one group is given breast screening, the other are not, and the end results are compared.

I think this is better, mainly because it reduces bias best. Bias – skewing of your results – can happen when, for example, the healthiest people (and least likely to die of breast cancer) attend most for screening; it will look as though screening made them live longer, but in fact, they would have lived longer anyway.

Many case control studies will try and account for this by controlling mathematically for these types of bias. But it may not be easy to do this fairly.

We can see this with this example, which I have taken from Karsten Jorgensen’s work on this. The Malmo trial (in Sweden) contained over 21,000 women in each of the screening and non screening groups, aged over 45, and followed for a mean of 8.8 years. The data were assessed as a RCT, and the difference in mortality between the two groups was negligible. But the same data was then assessed as though it was a look-back, case control study, and found an odds ratio of 0.42.

The Malmö mammographic screening trial assessed as a randomised trial (Janzon and Andersson, 1991)

Invited (n=21 088) Controls (n=21 195)
Proportion 50% 50%
Breast cancer deaths 63 (31 were non- participants) 66
Mortality rate 0.299% 0.311%
Relative risk 0.96 (95% CI 0.68–1.35)

 

The Malmö trial assessed using a case-control design Janzon and Andersson, 1991)

Living controls (random sample) Women dead from breast cancer Total
Participation in screening
 Yes 229 36 265
 No 71 24 95
Total 300 360
Crude odds ratio 0.46
Adjusted (matching for age) odds ratio 0.42 (95% CI 0.22–0.78)
So in the case control design of the same Mamlo data, it seemed that you were much less likely to die of breast cancer if you had been screening. But we know when we analyse it as an RCT – as the trial was actually run –  that same large benefit does not appear.
I think this is very important because it demonstrates how we can be misled by supposed benefits from case control studies, but the RCT results are much less impressive in terms of a mortality difference.
This kind of problem isn’t new, but we seem to keep doing it. For example, a couple of years ago this study found that women over 80 who were diagnosed with early breast cancer were more likely to have had mammography. Given what we know about overdiagnosis (being diagnosed with a cancer at breast screening that was not going to interfere with your lifespan) or lead time bias (being diagnosed with a breast cancer earlier because of screening, but not in a way that would allow for better treatment) this wouldn’t be surprising.
In fact, the WHO have said  that we should not be using case control studies to assess the effect of screening; “observational studies based on individual screening history, no matter how well designed and conducted, should not be regarded as providing evidence of an effect of screening”.
We seem to be in a bit of a loop when it comes to breast cancer screening reporting. The bottom line is that RCTs are more reliable than case control studies, and it is either brave or daft to claim the difficulties with the evidence are resolved on lower quality data.

5 Responses to “Breast screening: case control vs RCT – the problems”