This report from Price Waterhouse Coopers was produced for the DoH in January. It was greeted with all kinds of publicity around the claim that technology could save the NHS billions.
One line in the report caught my eye a couple of months ago which I knew was wrong – it, together with reference 3.1 and the correspondence with PWC are noted below. It made me wonder just how much of this report – which the DoH have told me after an FOI request cost over 75K – is based on fact.
Below are notes on the quality of the evidence PWC have used to produce what they call evidence which Department of Health say has “confirmed the potential for resources in the order of billions of pounds each year to be reutilised for better care by the NHS from making better use of information and technology.” I think the report did nothing of the sort.
My justification for this is below. The next time it’s claimed that this report showed how much technology could ‘save’ the NHS money, I’d suggest asking for evidence and checking the references.
Dr Michael Power has very kindly edited my typos and bad spelling, but any errors remaining are mine.
The following PWC quotes are from “A review of the potential benefits from the better use of information and technology in Health and Social Care” (link above). My comments follow each excerpt.
PWC review Page 23: Summary of supporting evidence: Clinical decision support in secondary care leading to a reduction in avoidable Adverse Drug reactions
“A more recent report in 2008 estimated the cost of ADR related admission to be c.£1.9bn p.a. This report used the same assumptions as a 2004 study (ref 1.1) and assumed that 6.5% of hospital admissions are the result of adverse reactions, resulting in an average stay of eight days). (ref 1.2)
…
The introduction of an electronic prescribing solution by Doncaster Royal Infirmary was found to have the potential to result in a 60% reduction in ADRs (ref 1.4)
Based on the assumption that an ePrescribing service could reduce preventable ADRs by c.60% (based on case studies – ref 1.4) and using total NHS costs of ADR related admissions of c£1.9bn, of which 50% are preventable (ref 1.3), an ePrescribing service could result in cost benefits of c.£570m p.a.”
MM comments
The first study (ref 1.1 [Adverse drug reactions as cause of admission to hospital: prospective analysis of 18,820 patients, British Medical Journal, July 2004]) did not examine how many ADRs it was possible to reduce while still having benefit from the drugs, ie “many of the implicated drugs have proven benefit”. There is no doubt that many prescribed drugs have adverse effects. But the study did not claim that these can all be negated.
Reference 1.2 [Compass study, 2008] was referenced to a Guardian story http://www.guardian.co.uk/society/2008/apr/03/nhs.drugsandalcohol. This in turn references a report from the ‘thinktank’ Compass, which has a press release on its website on the same date. http://www.compassonline.org.uk/news/item.asp?n=1551. This states that it is running an ongoing investigation which is due to be published in the Autumn. It appears therefore that PWC used only the Guardian story and the press release on the Compass website (which is unreferenced) to support their claim.
PWC’s report assumes that an ePrescribing service could reduce admissions for preventable ADRs by c.60%. This is based on (ref 1.3 [http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0033236])
This study was a high quality meta-analysis. However, it did not examine people who were outpatients or had an emergency visit to hospital. Additionally, the study regarded an ADR as preventable when “the drug event was due to a drug treatment procedure inconsistent with present-day knowledge of good medical practice or was clearly unrealistic, taking the known circumstances into account” . Evidence most relevant to the NHS is from 2 of the 8 studies conducted in the UK, where the NHS issues guidelines on treatments via NICE and SIGN.
These two studies are:
http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2710.2006.00744.x/abstract
and
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0004439
The first study was a pilot for the second study. The second study noted that “half of ADR were definitely or possibly avoidable”. It did not attempt interventions to test this hypothesis, and offered no evidence that it is possible to reduce the burden of adverse drug reactions.
Evidence that it is possible to reduce the burden of adverse drug reactions was claimed by the PWC report to be provided by a study at Doncaster Royal Infirmary in which electronic prescribing had the potential to reduce ADRs by 60% (ref 1.4 [ “JAC Medicines Management Case Study”]). This seems to be an unpublished report. However, some searching found that JAC are a company claiming to the “The UK’s medicines management specialists” and had participated in a case study in Doncaster. After trawling through the JAC website for some time I found this article
http://ccgi.jacpharmacy.plus.com/joomla/Doncaster%20Royal%20Infirmary.pdf
and
This JAC document says that “ A further audit of the effect of electronic prescribing and associated clinical decision support has shown that the system can reduce potential adverse drug events by up to 60 per cent”. But no data are presented to support the claim.
The PWC report then claims that an “ePrescribing service could result in cost benefits of c.£570m p.a.”. This assumes that it works; but we haven’t been shown any evidence to support this.
With this assumption, and noting that many NHS hospitals do not yet have ePrescribing, PWC then “conservatively” estimate potential NHS total gross benefits of c.£285m p.a. With some hand-waving about the costs of implementing ePrescribing in 50% of NHS hospitals in 1 year, they magically produce bottom line estimates of net benefits of c.£208m in year 1 and c.£270m p.a. from year 2 onwards.
This is quite amazing. There is incredible leapfrogging of imagination over non-knowledge to create a figure based on supposition. This conclusion is plainly daft.
PWC report Page 27: Summary of supporting evidence: Improved patient recovery during hospitalisation
“By providing patients, family members and social care workers with access to an online portal with increased educational information throughout the post-operative process, there could be significant post-discharge benefits for patients. These benefits can be allocated to key post-discharge objectives that indicate a successful end to care:
1. That the level of medical support provided, the number of follow up appointments and readmission rates fall within thresholds
• One of the major goals of the NHS is to reduce the rate of emergency readmissions as they are both costly and impact patient recovery
• According to NHS statistics there was 561k emergency readmissions to the NHS within 28 days of discharge in 2010/11
• A Department of Health funded report by the RAND corporation found evidence that suggested 15-20% of total emergency admissions within 28 or 30 days of discharge from hospital may be avoidable (ref 2.2)”
MM comments
RAND are a “non profit institution that helps improve policy and decisionmaking through research and analysis”. The cited report as (ref 2.2 [http://www.rand.org/pubs/technical_reports/TR1198.html]) is a “rapid review of systematic reviews”. It states that most of the analysed studies take place outwith the NHS, and that “prospective studies are needed to assess the proportion of readmissions that are avoidable in the contemporary NHS”. It also notes that readmissions are more common in people who are sicker.
Some of the criteria for avoidable admission are plainly unrealistic. For example, any readmission for the same diagnosis for a patient initially admitted with heart failure is treated as ‘avoidable’, yet heart failure is a chronic and usually worsening condition where future exacerbations are perhaps reducable but unavoidable. It is ironic that this section (about undertreatment) follows the pages about overtreatment (complications of prescription medication.) Specifically, social problems or a lack of community care was mentioned frequently as a cause of potentially avoidable admissions. There was no mention of ‘technology’ underuse as a cause for readmissions.
PWC report Page 27
•” Whilst unplanned readmission may be a result of a range of factors (infections, complications, choice of surgical approach and patient factors), expert opinion has highlighted a lack of patient education as a reason for readmission. A 2012 US study on readmission reasons post-pancreaticoduodenectomy found that poor discharge education was a key factor in high rates of readmission, specifically for dehydration and malnutrition (ref 2.3)”
MM comments
(ref 2.3) can be found at http://ons.metapress.com/content/f387578115952008/?genre=article&id=doi%3a10.1188%2f12.ONF.408-412
Pancreaticoduodenectomy is an uncommon and very complex surgical procedure which is relatively high risk for complications. This US study cited looked at 62 patients with pancreatic cancer who underwent this operation and documentation about ‘self care advice’ in the notes. Critically, this study did not trial the effect of discharge education – it’s easy to see that this could be a confounding factor rather than a causative factor (ie a busy ward where the nurses don’t tick the box that advice was given may also not do other tasks which could impact on the person’s health.)
PWC have not provided any RCT evidence that increased ‘education’ can reduce unnecessary re-admissions.
PWC report Page 27
•” It is estimates (sic) that by increasing the availability of educational material for patients to assist self management of post-operative care, there could be a significant reduction in emergency admission rates”
MM comments
PWC have provided absolutely no evidence to justify this statement. I would be interested to know who did the estimating, and how they did this.
PWC report Page 30: Summary of supporting evidence: Use of Acute operational performance information to drive contractual savings for commissioners
“While some / many are necessary, it is estimated that between 20% and 30% (ref 3.1) of short stay emergency admissions could be treated either at the A&E stage or even in primary care , and admissions are often made to avoid breaches of the 4 hour A&E waiting time threshold.”
MM comments
Ref 3:1: is “Primary Care and Emergency Departments, Report from the Primary Care Foundation, March 2010” and can be found at http://www.primarycarefoundation.co.uk/images/PrimaryCareFoundation/Downloading_Reports/Reports_and_Articles/Primary_Care_and_Emergency_Departments/Primary_Care_and_Emergency_Departments_RELEASE.pdf
This was the flaw I first spotted in this report on 20th January 2013 and I queried with PWC. On 4th February 2013 I had a reply which insisted it was correct:
PWC’s response of 4 February 2013
“You raised some queries re the PwC report for the Department of Health on use of technology, re references used. Sorry for the delay in getting back to you. The team have looked into this – here’s the details.
The reference 4.3 to the report on Primary Care and Emergency Departments (page 35) is correct. The PwC report refers to “up to 30%” of non – elective admissions” – reflecting the Primary Care report’s finding that “between 10%-30%” of emergency cases “could be classified as primary care” – ie “cases of a type that are regularly seen in general practice.”
One of the ways of achieving this would be to provide community teams with access to comprehensive patient data and clinical decision support including diagnostic services as highlighted in the report. While the report estimated potential financial benefits of various actions on IT, the overall financial estimates for potential savings did not include this specific action”
MM comments
In other words the response, not only insists that it was correct, but also infers that more technology could sort out the ‘problem’. I followed this up with another email.
PWC’s further response of 20 February 2013
“On page 33 of the report the word “admission” is incorrectly used instead of “attendance.” This is being corrected and the report should have read:
“Evidence suggests that certain non elective attendances could be avoided by managing patients in community care settings without compromising outcomes. One of the ways of achieving this would be to provide community teams with access to comprehensive patient data and clinical decision support including diagnostic services. Looking after patients at home is cheaper, mostly because there is no need for 24/7 medical and nursing, and where an overnight sitting service is required, this is substantially cheaper than a hospital ward.”
MM comments
So PWC confused admission – where people go into hospital to stay – with attendance – where people are seen by a healthcare professional as outpatients. Misleading, but they say they will correct it and send the DOH a corrected version. But they are still wrong. PWC have not provided data that says that attendances can be reduced – the study they cite is about where, or who, sees people who are sick. There is no data contained in this report that suggests that there is work that doesn’t need to be done. The relevant questions are “who does the work?” and “with what qualifications?” – in other words, emergency department staff versus general practitioner staff. The paper is about aligning the two, which is being done in many places already (two thirds, according to this report). PWC are wrong and in fact their statement makes no sense – these patients didn’t need ongoing management in the community, they needed a consultation with a healthcare professional out of routine hours.
Additionally, PWC are failing to support their assertion that there are no adverse effects of community care – this is unreferenced.
PWC report Page 30
“Bringing all short stay emergency admission rates down to the current national average would result in over 170,000 fewer emergency admissions with initial savings to commissioners amounting to over £130m p.a. (aggregated across all Commissioners nationally). Ultimately, if providers make operational adjustments to reflect the contractual changes, whole system benefits of a similar scale should be achieved”
MM comments
This is a startling and somewhat bizarre interpretation of ‘average’. Average admissions rates only tell you where the middle is, not whether it is best for patients. Do some hospitals under-admit, due to lack of beds or other pressures? Are admissions warranted, or not? Who is not being admitted? Who should be admitted?
The idea of this economic evaluation providing targets for clinical care would be laughable were it not so dangerous.
PWC report Page 31: Summary of supporting evidence: Reduction in Out-Patient (OP) procedures 0n a zero LoS In-Patient (IP) basis
“Once a patient is referred to a hospital and a diagnosis and treatment plan is decided upon, the follow -up care can often be carried out in a community or primary care setting rather than requiring repeated visits to hospital. It is estimated that between 20% and 30% (ref 3.1) of hospital follow up visits can be better managed in a primary care setting.”
MM comments
This is again referenced to the Primary Care and Emergency Departments, Report from the Primary Care Foundation, March 2010, and there is no data in it to support this assertion.
PWC report Page 33
“Evidence suggests that up to 30% (ref 4.3) of non elective admissions can be avoided by managing patients in community care settings without compromising outcomes. This is achieved partly by providing community teams with access to comprehensive patient data and clinical decision support including diagnostic services. Looking after patients at home is cheaper, mostly because there is no need for 24/7 medical and nursing, and where an overnight sitting service is required, this is substantially cheaper than a hospital ward.”
MM comments
Slightly unbelievably, reference 4.3 “Primary Care and Emergency Departments, Report from the Primary Care Foundation, March 2010”, is used – again!
PWC have made the same mistake again – which makes me think that this is not a one-off error, but a conceptual misunderstanding.
The report did not say that we could avoid admissions by treating patients in the community instead. These patients WERE NOT GOING TO BE ADMITTED IN THE FIRST PLACE; the 30% relates to people who could be seen by GPs rather than A+E departments. This means that all the calculations which follow this non evidence based statement crumble.
PWC report Page 34: Summary of supporting evidence: Clinical efficiency savings within Acute Trusts
“By provision of unified information views and availability of information to enhance clinical decision making
… a conservative saving of 3 minutes of nurse time would result in annualised savings to the acute trusts amounting to over 50 million
… A reduced number of duplicate diagnostics may be required. If it is assumed that 0.5% of outpatient attendances would not now require a simple (eg blood) test this would result in annualised savings to the Acute Trust amounting to over £15M (aggregated across all Acute Trusts nationally)”
MM comments
The same paper (ref 4.3) is cited yet again!
This is pretty amazing, given that there were no such workings out in the report being cited, and they haven’t demonstrated any such duplication of diagnostics or real-life savings on nurse time either. Wow.
PWC report page 34
“Through more integrated discharge management and better information availability for community staff, 30 day readmissions may be reduced (ref 4.1) “
MM comments
Ref 4.1 is “Northern Ireland Electronic Care Record – Proof of Concept Project Evaluation Report, 21/02/2011”
All I can find is some press releases http://www.northernireland.gov.uk/index/media-centre/news-departments/news-dhssps/news-dhssps-290412-poots-announces-multi.htm which announce the contractor, Orion, as the new caterer of electronic records in NI. I can’t find any data in Pubmed about this project. I have written to the http://www3.hscni.net/ecr/ system managers here to ask for this data. As it stands, there seems no publically available information which backs up PWC claim that electronic records can reduce readmissions.
The PWC report then goes on to discuss ‘supporting case studies’, which I don’t think they can be considered to be.
PWC report Page 43: Action 1; Case study 1: ePrescribing systems reducing Adverse Drug Reactions (UK)
MM comments
This case study was produced by JAC Medicines Management, a private company who wrote software used in the ‘case study’ in Doncaster.
PWC claim that this “could reduce potential adverse drug events by up to 60%.”.The link PWC provide to the research doesn’t work. On the JAC website there is this webpage:
which claims “After the introduction of e-prescribing and CDS on a four-ward medical unit at Montagu Hospital, Doncaster and Bassetlaw Hospitals NHS Foundation Trust, compliance with trust policy for recording of drug administration rose from 65% to 100% and potential adverse drug events (events with the potential to cause harm, delay recovery or result in a lack of control of symptoms) were reduced by over 61%”
This is referenced to “Barker A, Kay J. Electronic Prescribing Improves Patient Safety — An Audit. Pharm J 2007; 14:225”.
It doesn’t appear on PubMed. It does appear on this page of the Hospital Pharmacist webpage
A kind twitter friend sent me the full pdf. If you are a member of the Royal Pharmaceutical Society, it’s here http://www.pjonline.com/libres/pdf/hp/200707/hp_200707_audit.pdf. I am willing to bet that it wasn’t read in full by the authors of the PWC report . The audit found that adverse drug reactions (which they haven’t stated how they defined or measured) were reduced after the e-prescribing system was introduced – but it is impossible to draw conclusions about causation, since it is likely that with the new system being introduced staff changed their ways of seeking information about medication – and additionally, electronic prescribing was found to increase errors in selecting the correct medication from drop down menus.
PWC report Page 43: Action 1: Case study 2: Adverse Drug Reactions prevalence as reason for hospital admissions
MM comments
This is referenced to http://www.bmj.com/content/329/7
We already know that ADRs cause harm. What we have less data on is how to prevent them within the NHS (which works to different terms compared with eg North America.)
PWC report Page 46: Action 2: Case Study 1 Increasing patient educational information to reduce post-operative pain
MM comments
This cites a study showing that online advice about pain management could reduce pain experienced by patients post operatively. The link PWC cites is broken, and since they don’t write it out as references usually are, it takes rather longer to track it down than is necessary. But it appears to be this one,
http://pubmedcentralcanada.ca/pmcc/articles/PMC2232814/pdf/procamiasymp00004-0817.pdf
This study compares two groups who were both given the address of a website for information after surgery. One group got extra information about pain management, and the other one didn’t The patients who got more information about pain management had less severe pain. But two major problems; firstly, that the pain scores when patients arrived at home (before looking at the website) was quite different between the two groups, and second, there was no comparison with non electronic paper based or verbal information. Using this study to infer anything about how information is best given would be entirely misleading.
PWC report Page 47: Action 2: Case study 2: Health Unlocked – Royal National Orthopaedic Hospital (2011)
“Benefits include: … Increases patient satisfaction with 95% of patients prefer the new online process to the traditional pen and paper method; than and using traditional methods of reporting.
…
The process also encourages patients to report more often, which can lead to the identification of post-operative complications faster than follow up appointments.”
MM comments
Certainly this is what Health Unlocked, a company who offer data management, say: http://www.healthunlocked.com/about/in-clinic-casestudies/
As do the Royal National Orthopaedic Hospital in a press release;
http://www.rnoh.nhs.uk/home/news/nhs-patients-have-the-last-word-the-success-their-treatments
Interestingly there is a puff quote from ex health minister Andrew Lansley there, saying that “This is a great example of how healthcare innovation can make a real difference to patients’ lives. We want to modernise the NHS because we believe that patients deserve the best and most innovative treatments available. We are committed to creating an environment in the NHS where new medical technologies like this can flourish”
Quite clearly, the minister was a fan of the technology – what’s far less clear is the evidence of benefit. The trial is still ongoing, meaning that we don’t know if it works – or if there are any unintended harms. I’ve contacted Health Unlocked to ask them for further details.
PWC Page 48: Action 4: Case study 1: Paperless systems increasing clinical inefficiencies (sic)
“Objectives of the trial was (sic) to eliminate time and cost associated with paper notes e.g.:
• 50% of records were not available when requested (either due to loss or use by other care provider); and
• Clerks spent 60% of their time chasing missing case notes”
MM comments
I think they have made a typo with ““Paperless systems increasing clinical inefficiencies” – they are meaning to argue FOR paperless systems and more tech to be MORE efficient, not less efficient.
Unfortunately, and yet again, data isn’t available to examine these claims. As reference, PWC give two items
Again, a private technology company was contracted to provide this new database management service. The news stories seem to have been triggered by Alison Clare, who managed the electronic record at the trust when speaking about it at HC2011 informatics conference in Birmingham. Without the full data we can’t know whether the time efficiency was measured merely in finding the case notes or in using them – and given the multiple complaints from front line healthcare staff about the unwieldy nature of electronic medical records, we can’t be sure whether this ‘advance’ actually is one.
PWC report Page 49: Action 4: Case study 2: Paperless systems increasing clinical inefficiencies (sic)
MM comments
This is quite funny – they’ve made the same, own goal, typo AGAIN with “Paperless systems increasing clinical inefficiencies”.
Their links to news reports are broken, and I am presuming that as they link to news reports rather than to actual data, they haven’t seen any raw data. I think they mean to link to
http://www.businessweek.com/stories/2004-07-06/a-paperless-health-care-system
which is a piece saying things like “It’s very early, but Neaman says the results are promising. Doctors now receive patients’ mammogram test results in a single day, compared to a three-week wait before.”
Yes – it’s very early. And it states that it cost 5 million on staff training and 16 hours each before they could use it. That’s a big spend on technology.
PWC report Page Action 4: Case study 3: Evidence of appropriate patient placement
MM comments
This seems to have been included to make the point that some patients using A+E can be seen by primary care doctors. But, we already knew that! That’s why lots of out of hours services have the two (A&E and primary care) on the same site.
PWC report Page 51: Action 4: Case study 4: Potential to reduce Ambulatory Care Sensitive Conditions emergency hospital admissions
MM comments
This is based on a King’s Fund report
This report rests on the assumption that there are conditions “for which effective management and treatment should prevent admission to hospital” (page 3).
This is especially interesting to me, as for part of the GP contract, I now have to analyse a selection of our practice’s admissions to hospital and essentially justify them. Essentially I have found that patients who are admitted to hospital are admitted for good reason and not lightly.
The King’s Fund report confusingly seems to use “prevent” and “limit” as interchangeable when referring to admission.
It pronounces that admissions from an amazing range of conditions are essentially preventable, listing
Vaccine-preventable: Influenza and pneumonia; Other vaccine-preventable conditions
Chronic: Asthma; Congestive heart failure; Diabetes complications; Chronic obstructive pulmonary disease (COPD); Angina; Iron-deficiency anaemia; Hypertension; Nutritional deficiencies
Acute: Dehydration and gastroenteritis; Pyelonephritis; Perforated/bleeding ulcer; Cellulitis; Pelvic inflammatory disease; Ear, nose and throat infections; Dental conditions; Convulsions and epilepsy: Gangrene
There is a real dichotomy here. On one hand right at the start of the PWC report, we are advised that many patients have adverse drug reactions and are thus overtreated. In this part of the document, we are being advised that more treatment can prevent hospital admissions.
This is simply incorrect. Firstly, we do not have means which can tell us which patients, precisely, benefit, and which will be harmed through evidence based interventions.
Secondly, the idea that this list contains preventable conditions is laughable. For example, the King’s Fund say
“Many of these cases of influenza and pneumonia are vaccine-preventable.”
I’d suggest reading Cochrane on this http://summaries.cochrane.org/CD004876/vaccines-for-preventing-seasonal-influenza-and-its-complications-in-people-aged-65-or-older
And http://summaries.cochrane.org/CD000422/vaccination-for-preventing-pneumococcal-infection-in-adults.
As for the idea that doctors admit patients with gastroenteritis, gangrene, anaemia, or bleeding ulcers when they were either preventable or treatable in the community – this is neither evidenced or takes account or whether it was good for the patient.
PWC report Page 52: Action 4: Case study 5: Deployment of a healthcare-economy-wide portal
This covers a pilot study, http://www.bmj.com/content/344/bmj.e3529, whose methods and results have not been published.
To suggest that the outcomes – including a reduction in emergency admissions, outpatient referrals or ‘releasing 10-12 million a year’ can be measured at this point or even caused by the piloted intervention is hype and misleadingly conflates aims with achievement .
PWC report Page 53: Action 4: Case study 6: Trial of an healthcare economy-wide Electronic Care Record
MM comments
Data requested, not published online,
PWC report Page 57: Action 4: case study 7: Creation of an integrated health and social care team
MM comments
This case study is based on this King’s Fund report “Torbay Care Trust Integrated Health and Social Care (2011)”
http://www.kingsfund.org.uk/publications/integrating-health-and-social-care-torbay
The reported outcomes include:
– Reduced bed occupancy (daily average number of occupied beds fell from 750 in 1998/99 to 502 in 2009/10);
– Reduced emergency bed day usage (emergency bed day use for people aged over 75 fell by 24% between 2003 and 2008 and for those aged over 65 is 1920 per 1000 population compared to an average of 2698 per 1000 in 2009/10); and
– Delayed transfers of care from hospital have reduced to low level
This is all very encouraging, but it is not clear what the relevance is to PWC’s “Action 4 – Provision of complete and accurate clinical and attendance information to clinicians and carers at point of care via digital portals or other similar solutions”. Perhaps the lack of relevance is why it was not referenced on page 33.
The King’s Fund report does say that “The work of co-ordinators was underpinned by a commitment to sharing data, enabling co-ordinators to access information about users from the hospital, general practices and the care trust.” But, the impact of integrating information systems was not assessed. Rather, it was seen as a milestone in service integration.
The bottom line is that the PWC do not do what they or the Department of Health claim, which is that the report “confirmed the potential for resources in the order of billions of pounds each year to be re utilised for better care by the NHS from making better use of information and technology.” This report is bad quality. It should be dismissed, and the DoH should be spending their money more wisely.
3 Responses to “Price Waterhouse Coopers report on technology was expensive and is flawed”