NEJM Alzheimers study: all it seems?

The study published yesterday has made the headlines across the media;

” The study they funded, led by Professor Robert Howard from the Institute of Psychiatry at Kings College London, and published in the New England Journal of Medicine, has concluded that the drugs carry on working in people whose illness has become severe.

“For the first time, we have robust and compelling evidence that treatment with these drugs can continue to help patients at the later, more severe stages of the disease,” he said. “We observed that patients who continued taking donepezil were better able to remember, understand, communicate and perform daily tasks for at least a year longer than those who stopped taking the drugs. These improvements were noticeable to patients, their caregivers and doctors.”

I’m not quite so sure. The trial took 295 patients living in the community who had moderate to severe Alzheimers disease and who were already on donepezil. They were divided into groups;

treated with donepezil,

discontinue donepezil,

discontinue donepezil and start memantine,

continue donepezil and start memantine.

They were followed for a year. At the start, in the ‘methods’ section, the researchers say

” The minimum clinically important differences were 1.4 points on the SMMSE and 3.5 points on the BADLS.”

These are the’standardised mini mental state examination’, and ‘Bristol Activities of Daily Living’. The latter is a functional assessment and a better guide for useful improvements.

The important bits in the results are in a box that I can’t manage to cut and paste, but the relevant bits are

BALDS – Difference between scores for continued vs discontinued donepezil, which were -2.9 at 52 weeks, with an ‘overall difference’ of -3.0.

Active versus placebo memantine scores were different by -1.5.

The BALDs score is a 60 point scale. The authors told us in the discussion that a 3.5 point difference would be clinically significant, but the difference in the results is a points score on this scale of 3. What does this mean? I am not sure, because small improvements in function may be very clinically meaningful (eg the ability to keep up with personal hygiene).

The other thing that bothers me is this.

“The study was overseen by King’s College London and was funded by the U.K. Medical Research Council (MRC) and the Alzheimer’s Society.”

and ”

Recruitment was slower than anticipated, and it was not possible to extend the recruitment period, since the public funder of the study (MRC) believed that the disadvantages of a delay in reporting results outweighed the benefits of increasing the power of the study.”

This is quite unusual, because a better powered study usually means more definitive results. Around 50 patients in each of the 4  groups who completed the study. I’m not sure this is definitive enough evidence to make a large change in practice. Additionally, patients were excluded who were felt unable to “adhere to the study regimens”. This may have produced a bias – the cognitively best or most socially supported may have been over-represented and may not reflect normal practice.

The other note is the amount of potential conflicts of interest.

“Dr. McShane reports receiving payment for work as the local principal investigator for commercial trials from Abbott, Novartis, i3 Innovus, and Medivation; Dr. Lindesay, receiving consulting fees from Novartis NEURONET and lecture fees from Janssen, Novartis, Eisai, and Pfizer; Dr. Ritchie, receiving consulting and lecture fees from Pfizer and Eisai and grant support and reimbursement for travel expenses from Eisai; Dr. Barber, receiving royalties from Arnold Press; Dr. Burns, receiving royalties from John Wiley; Dr. Findlay, receiving lecture fees and reimbursement for meeting expenses from Eisai, Pfizer, and Lundbeck; Dr. Jones, receiving consulting fees from Merz Pharmaceuticals and Janssen, grant support from Eisai, Lundbeck, and Merz Pharmaceuticals, and lecture fees from Lundbeck, Merz Pharmaceuticals, Eisai, Pfizer, and Novartis and being a board member of Merz Pharmaceuticals, Lundbeck, Eisai, Pfizer, and Lilly; Dr. McKeith, receiving lecture fees from Novartis; Dr. O’Brien, receiving consulting fees from GE Healthcare, Bayer Healthcare, and Servier and lecture fees from Pfizer, Eisai, Novartis, Lundbeck, Eli Lilly, Shire, and GE Healthcare; Dr. Passmore, receiving consulting fees from Pfizer, Lundbeck, Novartis, Shire, and Johnson & Johnson and lecture fees and reimbursement for travel expenses from Lundbeck and Pfizer; Dr. Katona, receiving consulting fees from Lundbeck and Eli Lilly, grant support from Lundbeck, lecture fees from Lundbeck, Lilly, Shire, and Pfizer, payment for development of educational presentations from Lundbeck, and reimbursement for travel expenses from Pfizer and being a board member of Lundbeck; Dr. Ballard, receiving consulting and lecture fees from Lundbeck, Eisai, Bristol-Myers Squibb, Janssen, Acadia, and Novartis and grant support from Lundbeck and Acadia; Dr. Brown, receiving lecture fees from UCB Pharma, GlaxoSmithKline, Solvay, and Lundbeck; Dr. Banerjee, receiving grant support from Pfizer, lecture fees from Lundbeck, and reimbursement for travel expenses from Pfizer and Eisai; and Dr. Bentham, receiving consulting fees from TauRx Therapeutics.”

This work is interesting, but it didn’t achieve the level of functional benefit the authors had said would be clinically significant. I’m wary of false hope being generated.

6 Responses to “NEJM Alzheimers study: all it seems?”

  1. deeohhemm March 9, 2012 at 4:54 pm #

    A quick look at the paper describing how the investigators arrived at their minimum clinically important differences (MCIDs) for the two outcome measures hardly fills one with enthusiasm for the change in practice being promoted.
    All seems to have been a bit subjective and post-hoc. Eminence Based Medicine anyone???

  2. steve March 12, 2012 at 4:13 pm #

    I share your scepticism.
    The Bristol ADLS questionnaire was first written up in the journals in 1996 I think.
    In 2009 a review of various ADL scales used in research was published:

    J Neurol Neurosurg Psychiatry2009;80:7-12 doi:10.1136/jnnp.2008.155838

    here’s is an abstract of that paper, the conclusion of which is important:


    Background: Instrumental Activities of Daily Living (IADL) questionnaires can be helpful in diagnosing dementia and are often used for clinical follow-up and treatment evaluation in dementia patients. Despite the large number of questionnaires, their quality has received little attention.

    Objective: To systematically review the measurement properties of all available structured informant-based (I)ADL questionnaires, developed or validated for use in demented patients.

    Methods: A systematic literature search was conducted in MEDLINE, PsycINFO and EMBASE for psychometric articles on (I)ADL questionnaires. In addition, reference lists of all retrieved articles were screened. Standardised criteria were used to assess the quality of the measurement properties. When possible, investigators were contacted to obtain missing information. Two authors independently extracted studies and performed the quality assessment of the questionnaires.

    Findings: Thirty-two articles were selected, covering 12 (I)ADL questionnaires. Information on 52.3% of the quality aspects was not available, 32.4% of the ratings were indeterminate, 8.1% were positive, and 7.2% were negative. Out of eight measurement properties, two scales (the DAD and the Bristol ADL) received two positive ratings and were classified as of moderate quality. Five scales (ADL-PI, ADL-IS, B-ADL, CSADL and Lawton IADL) received one positive rating.

    Interpretation: The findings indicate that improvements in and more data on psychometric properties of (I)ADL questionnaires for dementia patients are necessary in order to justify their use.

    I think it would be higly wrong to change prescribing practice for drugs like memantine and donepezil on the basis of this single paper.

  3. Julie March 13, 2012 at 4:00 pm #

    From my own totally unscientific point of view, the results of this study mirror my mum’s experience. She had Alzheimers and she stayed on Aricept for several years through the progression of the illness. Her decline was much slower than others who had the illness and there was also an absence of the more distressing symptoms such as depression, hallucinations and aggression. I take your point about how the study has been conducted and I think they would have to do a more thorough one with stricter parameters, but I do think there’s something in what they’re saying.

  4. Deebles March 15, 2012 at 2:03 pm #

    This is a problematic study.

    The groups used per arm were rather small, meaning that, as Richard Lehmann points out in his response to my comments on his blog post about this, the study was barely powered to find any results of clinical interest The limited follow-up is unfortunate, but unsurprising – at the end of the year, 52/72 patients in the double-placebo arm had dropped out (see figure 2). And as rather less dropped out in the intervention arms (e.g. 34/72 in the double-intervention arm), the findings may well have been biased towards the null, as those who dropped out seemed to be doing rather worse, and as only 47% of those who dropped out from treatment (64 / 137) were assessed at the 52-week point. (As discussed by the authors in the appendix

    Overall: good statistical evidence for a borderline clinical benefit of donepezil vs placebo; borderline statistical evidence for a very slight further benefit of adding memantine. And this finding would seem to be roughly consistent across all scales measured. The strongest evidence of benefit almost seems to be the increased capacity to stay within the trial among those in a treatment arm… but this, of course, was not a study endpoint in itself. And I’d agree that one note of caution would be that picking only those thought more likely to adhere at the start smells slightly of cherry-picking. (I would have liked to know how many were excluded, to really evaluate how important this might be, however).

  5. Margaret McCartney
    margaretmccartney March 17, 2012 at 10:53 pm #

    thank you – Julie I do take your point, however from the point of view of the study, the effect was small overall – that may still be important – but from the way the paper has been presented, I am afraid that it will give false hope to many.

  6. Jim Thornton April 24, 2012 at 9:39 pm #

    I think you’re all too generous. I think they cheated. The first reduction in sample size makes no sense at all and the second is damn near an admission that they peeked at the data. Details here
    But you’re spot on about the trivial difference even if true.