The Daily Telegraph reported yesterday that “Breast cancer screening ‘works and we should move on” – “Women should undergo breast cancer screening because it halves the chance of them dying of the disease, according to a new study that claims to draw a line under the controversy”.
So simple? No!
Here’s the paper. It’s a case control study. So you look back at women who have died of breast cancer, and take a comparable group of women, but who didn’t die of breast cancer. Then you find out if one group went for breast screening more than the other. The researchers found that there was ”an average 49% reduction in breast cancer mortality for women who are screened.”
I don’t think this means that we have proven whether breast screening works or not. The best way to do that is through a randomised controlled trial. In this type of trial, a comparable group of women are sorted into two random groups, one group is given breast screening, the other are not, and the end results are compared.
I think this is better, mainly because it reduces bias best. Bias – skewing of your results – can happen when, for example, the healthiest people (and least likely to die of breast cancer) attend most for screening; it will look as though screening made them live longer, but in fact, they would have lived longer anyway.
Many case control studies will try and account for this by controlling mathematically for these types of bias. But it may not be easy to do this fairly.
We can see this with this example, which I have taken from Karsten Jorgensen’s work on this. The Malmo trial (in Sweden) contained over 21,000 women in each of the screening and non screening groups, aged over 45, and followed for a mean of 8.8 years. The data were assessed as a RCT, and the difference in mortality between the two groups was negligible. But the same data was then assessed as though it was a look-back, case control study, and found an odds ratio of 0.42.
The Malmö mammographic screening trial assessed as a randomised trial (Janzon and Andersson, 1991)
Invited (n=21 088) |
Controls (n=21195) |
|
|---|---|---|
| Proportion | 50% | 50% |
| Breast cancer deaths | 63 (31 were non- participants) | 66 |
| Mortality rate | 0.299% | 0.311% |
| Relative risk | 0.96 (95% CI 0.68–1.35) | |
The Malmö trial assessed using a case-control design Janzon and Andersson, 1991)
| Living controls (random sample) | Women dead from breast cancer | Total | |
|---|---|---|---|
| Participation in screening | |||
| Yes | 229 | 36 | 265 |
| No | 71 | 24 | 95 |
| Total | 300 | 360 | |
| Crude odds ratio | 0.46 | ||
| Adjusted (matching for age) odds ratio | 0.42 (95% CI 0.22–0.78) | ||
Thanks Margaret. I still don’t understand what they did in this case-control analysis and I can’t access the original. There is definitely a typo in that table though as 36+24 =60 not 360.
Are we agreed that the only factor is selection bias though? That was the main thing we were discussing yesterday.
it’s total stringing along – so 300 + 36+ 24 = 360 – confusingly written
yes, indeed, my fault – I was presuming that it was the same problem as the second study down – ie more mammography= more cancer – my fault for having not read the full paper first. sorry!
yes, I agree it would be better if the full workings were placed out, and not behind book covers (you can search inside to find it – which I had to do as I can’t find the book) but the problem is that this has hardly been done outside Mamlo – I am not aware of any other RCTs that then went back and did a case control using the same end data – it would be possible to do (but not easy to blind the researchers)
Why would you need to blind the researchers? The outcome in the cases is death from breast cancer, which can be established from registries and the exposure is participation in screening. Is there much risk of classification error?
Thanks again
AM
think more that researchers should be blinded to the result of the RCT, so that if using same data with case control method they don’t know what the other results were….
Thank you, Margaret, for explaining this issue so clearly. A newpaper article rarely provide thorough explanations, so some additional detail is warranted.
This is a difficult topic – how can a 50% reduction in the
risk of dying from breast cancer be all down to bias? I think some
additional background numbers may make it seem less odd.
The authors found that 2 051 of 3 650 (56%) of controls had been screened, whereas the corresponding figure for cases was 146 of 427 (36%). The difference in attendance was thus 20 percentage points, which is much easier to understand
could be due to self-selection bias. These numbers were the basis for the calculated “effect” of screening.
There is a couple of additional peculiar issues in this paper.
1. The opening sentence is a statement that the randomised trials showed a 25% reduction in breast cancer mortality. The reference provided is the 2002 WHO/IARC report on breast screening. The authors then go on to show that case-control studies have very, very consistently found a 50% effect, including their own new Australian analysis (49% for the meta-analysis and 52% for Australia, to be precise). They make no comment as to how the effect of breast screening in a real life setting is twice as large as that found in the rigorously controlles randomised trials. Differing attendance cannot be the answer, as there were no correlation between attendance rates in the trials and the estimated effect.
The Canadian trials, which only included those who accepted an invitation to
screening (essentially 100% attendance), could not demonstrate any effect.
2. The case-control studies are so consistent in their estimate of effect
that they vary by only a few percentage points if we exclude two outliers
(I2 for heterogeneity was 0.0%!) . For comparison, the randomised trials
varied between a 35% effect and no effect at all. Yet again, the authors
make no remarks about this remarkable inconsistency between their findings
and studies of much stronger design.
What they demonstrate with their study is not a huge and consistent effect
of breast screening. It is a huge and systematic error in the case-control
design. However, we already knew about this systematic error, because we
know from the randomised trials that when such studies are analysed using the
case-control design, they can show an “impressive”
58% effect, even when there is no effect at all, as demonstrated above by Margaret. A consistent 50% effect in such studies is then hardly impressive. This is why the very same 2002 IARC/WHO report that the authors of the new article start by quoting denounces case-control studies for the evaluation of the effect of breast screening, no matter how elaborate the design.