should Pharma teach students critical appraisal skills?

I think not.

Last week I was very pleased to have been asked to  the Glasgow Evidence Based Medicine Society. I going to talk about the problems of medical headlines in the media. I got there a bit early and so went to a session being run by Pfizer on how to critically analyse a paper. It was all quite standard stuff – apart from the wee Pfizer logo on the bottom of every slide – in terms of how to unpick a paper and analyse it’s flaws.

But what was missing, in an enormous, black-hole kind of way, was a discussion about the way in which pharmaceutical companies can choose not to publish clinical trial data. Despite the APBI claiming that criticism levelled at pharma is ‘fashionable’ they have not addressed the fact that actually, people like Sir Iain Chalmers have said as far back as 2004 that non publication of trials kills people. Not so recent a fashion, then. Ben Goldacre’s recent book Bad Pharma offers spades of evidence that despite assurances from Pharma that this kind of dangerous practice would stop, there is copious evidence that it hasn’t.  The discussion from Pfizer didn’t mention any of this or the way in which this publication bias distorts what we know about the evidence, leads to overestimation of pharmaceutical benefits, and unnecessary risks and harms for patients. Have a look at this paper which details some of the correspondence to and from managers at Astra Zeneca, and cringe.

And so, by the time it came for me to give my talk, I was ridden with angst. This enormous problem hadn’t been mentioned, and if there is to be any discussion about the need for evidence in clinical practice, it has to be about what we don’t get to know about. If we don’t know about all the clinical trials of a drug, we don’t know as much as we should, and any attempts to critically analyse the evidence will thus fail in making a fair judgement.

And so I apologise if I went on a bit, or was overly harsh, but until we start talking about these problems with evidence based medicine, we will continue to do more harm than we need to. Pfizer, if you want to talk to me, am happy to discuss, at .





13 Responses to “should Pharma teach students critical appraisal skills?”

  1. Graham Lappin October 25, 2012 at 1:37 pm #

    I’m an academic these days but spent many years in the pharmaceutical industry. I’m afraid it’s an undeniable fact that publishing data when you are pharma is notoriously difficult. Every pharma scientists knows it. In my view it’s all part of a systemic problem of business versus science. The same applies to the food and chemical industries. There seems to be a period of denial presently, but lets hope the facts will prevail.

  2. Paul Callan October 25, 2012 at 5:21 pm #

    In a similar way, should the newspapers publish when they have attempted to set up stings for MPs etc in an attempt to highlight their dishonesty, and in fact they prove to be incorruptible.

  3. steve October 25, 2012 at 9:03 pm #

    I don’t go to drug company sponsored anything. Ealier this year i was at a BMJ Masterclass in Glasgow. One session was “sponsored” (by Champix makers – the talk was on smoking cessation) and I opted out of that.

    Getting Pfizer to run a session of critical appraisal is a bit like asking a fox to show your hens round their new coop!

    But don’t worry Margaret your definitely one of the good people so won’t hold it against you 🙂

  4. martin ferry October 26, 2012 at 1:33 pm #

    This was an event organised by the medical students whom you teach. As a speaker at the event you’d have seen the programme in advance. Wouldn’t it have been more sensible to have advised your students in advance, rather than make a fuss on the day then blog about it afterwards?

  5. Margaret McCartney
    margaretmccartney October 26, 2012 at 2:53 pm #

    not teach directly. The programmes said it was a session on “Introduction into the practice of Evidence-Based Medicine ” -with no note of who was doing it

  6. martin ferry October 26, 2012 at 5:56 pm #

    Margaret, it is simply not true that there was “no note of who was doing it”. The programme on their website clearly states who’s presenting: have a look at the link you’ve already included in your blog.
    It’s not hard to identify the “Big Pharma” input, even someone who’s less exercised about it than you are can do so. As such I think it would’ve been better for you to have actually made a difference by using your University role to intervene to prevent this from happening if you believed it to be a bad idea, rather than making a drama about it, blogging about your objection to it, and (somewhat bafflingly to me) issuing an apology for your reaction to the company’s presence at the same time.
    According to your website you are ” undergraduate tutor at Glasgow University” and the undergraduate society whose conference you addressed is for all Glasgow medical undergraduates. I therefore don’t understand your statement that you don’t teach these students “directly”. How do you act as a tutor then? What sort of tutor doesn’t teach “directly”?
    You advocate transparency from others. Don’t we need some from you?

  7. Margaret McCartney
    margaretmccartney October 26, 2012 at 6:24 pm #

    I’m sorry, I should have said ‘there was no note that Pfizer was doing it’. The names of the presenters were there but not their affiliation as employees of Pfizer. In ‘who’ I meant Pfizer. I didn’t know or have any information prior to the event to say that Pfizer were involved.
    The presentation I did was nothing to do with my role as undergraduate tutor. I haven’t ever taught the students who are involved with the conference. (that’s what I meant by ‘not directly’. I have a group of around 8 per year, and am tutor to the same group all year.) I was the last speaker of the day. Otherwise there might have been time to do something else. I spoke for a few minutes about the problems of concealed data caused by pharma and then I gave my prepared talk.
    I am struggling to know what exactly you are accusing me of, in saying I have somehow failed to be transparent, so please spell it out. And if you have any conflicts of interest here, do let me know.

  8. Martin ferry October 26, 2012 at 6:51 pm #

    Thanks for the clarification. I’ve no conflict of interest nor particular axe to grind. I like your blog. You dissociate your presentation at the conference from your professional role as a tutor, and that’s fine. The only resultant question is if the presentations were dissociated from professional roles, why does the provenance of the “critical analysis” talk matter?
    That’s not to denigrate the issue of data suppression, and I’m sure the students would have appreciated your discussion of it, but it’s arguable whether that important topic falls within the scope of a short talk about “critical appraisal” of data.

  9. Margaret McCartney
    margaretmccartney October 26, 2012 at 7:12 pm #

    every other teaching session on EBM I’ve been to – including short sessions – has included the problems of publication bias, non publication, sponsor bias, etc. None of that was included in this session.

  10. Martin Ferry October 26, 2012 at 7:24 pm #

    This was a student-led event, held at the weekend, attended (I would assume) by a small proportion of the undergraduate medical students. The suppression of data is clearly an issue about which you hold strong views, and I commend you for that.
    I suppose the main point arising from your blog is whether this topic is covered in the curriculum, and delivered to all students, not just those who choose to pitch up at the weekend to an extramural event. As a tutor employed by the University concerned, do you know if this is covered in the curriculum, or do you plan to check? This would in my view probably be the most effective course of action to remedy the concerns you have raised.

  11. Margaret McCartney
    margaretmccartney October 26, 2012 at 9:37 pm #

    I will check what is formally written. There are other people who are revising the curriculum and who have responsibilities nationally (in terms of what the GMC expects for registration) and who would be better to tell me/know. I will let you know.

  12. Elaine Leung October 27, 2012 at 1:37 am #

    Thanks for speaking to us last Saturday. I founded the organisation in 2009 exactly because the curriculum doesn’t really cover this stuff formally (now they have a research SSC and planning to expand the curriculum).

    The first year running GEMS was really tough (there weren’t that many trainee-led research societies at the time). Help was limited from a few clinical academics. Even to date, we still find it difficult to get connected to more senior guys to help out in out-of-hours events at times. Medical schools in the UK are really recognising this gap and in the process of change, but as we all know curriculum change takes time.

    Anyways, the speakers were recommended by one of the clinical academics in Glasgow and we got in touch. To be honest, I wasn’t really sure what the workshop was about before I went. It probably should have been called “how to critically appraise a paper in 30 minutes?” instead, as that was what they were trying to do. They went through a modified checklist for critical appraisal using a selected paper (cranberry juice trial for UTI). Is that all what EBM is about? I think not.

    In addition, isn’t it like the real world? The research landscape is full of conflicting interests and advice. Thanks for making the post-conference conversation interesting.

    Just read @ajkeen CNN opinion on the vulnerability of Malala who has obviously become a sensational figure ( Should we protect medical students in the similar manner as Keen has suggested? Or, regard medical students as adult learners and provide them with different viewpoints (yours versus Pharma’s) to help them decide?

    Perhaps you would be interested in a public debate with a team from the industry at our conference next year? Let us know- it would be fantastic to facilitate that next year!

  13. JackBarnes December 5, 2012 at 10:56 am #

    Nice blogging section discussing by the blogger….its really helping us so thanks for sharing it.