Dr Foster, association and causation

The headlines today have been driven by publication of the ‘Dr Foster’ (“the UK’s leading provider of comparative information on health and social care services”) annual hospital guide. The Guardian have reported it as have the Telegraph, who  headline  “Experts warn hospitals not to cut staff over weekend death risk.”

They have presented their data as a spreadsheet. There is a lot of data, under the tab ‘Reducing mortality at nights and weekends’. What strikes me is that key data appears to have been collected by questionnaire via ‘trust submissions’ to the company, i.e. how many consultants and registrars were on site and on call at particular times of the day or night. I’m curious as to how reliable this is, as it seems to be data collected by administrators rather then clinical staff. Several hospitals appear to have no consultants on the wards at 11am on a Sunday. Another has 37 consultants on call and on site. This kind of variation, together with my personal experience, makes me suspicious that some numbers are wrong.  Was the variation checked, and how? There is no discussion section in the paper to tell us.

Bed numbers are not collected by direct counting on wards on the days being measured; rather the data is drawn from the NHS Estates Survey. Dr Foster have then produced ratios of staff to bed numbers. In the end they say that higher mortality is associated with less staff.

Not for a moment do I think that less staff is good for patients; I don’t think that cutting down on staff levels in hospitals is likely to be a good way to improve mortality, morbidity or care levels; just the opposite. What I am unconvinced about is whether the association – if it exists, I’m not really sure from the way the data has been presented – represents the cause of patients possibly dying more often if they have been admitted to hospital at the weekend.

There are alternative explanations. Weekends may be a time when more ill people end up in hospital. The association between higher mortality and weekend admissions has been found in some studies, for example here and here. What is less clear is why this is. Some studies have looked at the effect of providing different types of care, for example, in ‘acute admissions units’, such as here, and others have postulated that their lack of increased mortality at the weekends is due to similar units. Yet emergency care wards are now common in the UK; I don’t think Foster have proved that less staff at weekends cause higher deaths, I’m not even sure that we have demonstrated that less staff are available. At weekends staff don’t have outpatient clinics to attend to and may actually have a higher concentration of senior staff on the ward (not clinic) compared to weekdays. What about other explanations: are less unwell patients admitted during the week (for example, from clinic) which biases weekend admissions to a higher concentration of more unwell patients at the weekend?

I am also concerned because data is only as good as what goes in. As far as I can see, Dr Foster are a profit making company who take the data the NHS collects and then collates it in a way that is sold back to the NHS. This doesn’t strike me as good value for money. Superimposed on their hospital guide is a fall in mortality rates over the last 10 years with the  pictures of their annual reports above each stepwise fall. I fear that some may be drawn to feel there is an association there too. I think not.

2 Responses to “Dr Foster, association and causation”

  1. Anne Marie Cunningham November 29, 2011 at 6:20 pm #

    Thanks Margaret. I take your point about self-reporting but if anything this is likely to have reduced the strength of the relationships found. Is there any reason to think that the worst performing hospitals will under-estimate the numbers of staff available at weekends, whilst the best performing will over-estimate?

    With regards to the relationship with illness severity presenting at the weekends, this study by some of the Dr. Foster team does start to look at some of these factors by considering diagnosis and co-morbidity, though not severity of illness.
    http://qualitysafety.bmj.com/content/19/3/213.short

    Last link is this rapid review of the literature around week-end mortality http://www2.nphs.wales.nhs.uk:8080/HealthServiceQDTDocs.nsf/815f97434d281f8b80257508003d3986/7f113a4146140de1802578b1002d7781/$FILE/050711%20Excess%20WE%20deaths%20v1.doc
    Unfortunately we don’t know what to do about this problem.

  2. margaretmccartney December 1, 2011 at 7:38 pm #

    I am gathering data about this. be back with you.