What's the evidence that NICE guidance has been implemented? Results from a national evaluation using time series analysis, audit of patients' notes, and interviews

Abstract

Objectives: To assess the extent and pattern of implementation of guidance issued by the National Institute for Clinical Excellence (NICE).

Design: Interrupted time series analysis, review of case notes, survey, and interviews.

Setting: Acute and primary care trusts in England and Wales.

Participants: All primary care prescribing, hospital pharmacies; a random sample of 20 acute trusts, 17 mental health trusts, and 21 primary care trusts; and senior clinicians and managers from five acute trusts.

Main outcome measures: Rates of prescribing and use of procedures and medical devices relative to evidence based guidance.

Results: 6308 usable patient audit forms were returned. Implementation of NICE guidance varied by trust and by topic. Prescribing of some taxanes for cancer (P < 0.002) and orlistat for obesity (P < 0.001) significantly increased in line with guidance. Prescribing of drugs for Alzheimer's disease and prophylactic extraction of wisdom teeth showed trends consistent with, but not obviously a consequence of, the guidance. Prescribing practice often did not accord with the details of the guidance. No change was apparent in the use of hearing aids, hip prostheses, implantable cardioverter defibrillators, laparoscopic hernia repair, and laparoscopic colorectal cancer surgery after NICE guidance had been issued.

Conclusions: Implementation of NICE guidance has been variable. Guidance seems more likely to be adopted when there is strong professional support, a stable and convincing evidence base, and no increased or unfunded costs, in organisations that have established good systems for tracking guidance implementation and where the professionals involved are not isolated. Guidance needs to be clear and reflect the clinical context.

Figures

Fig 1

Wisdom teeth extraction activity, 1992-2001

Fig 2

Hospital use of paclitaxel (top) and docetaxel (bottom)

Fig 3

Total number of implantable cardioverter defibrillators implanted by quarter, 1995-2001

Fig 4

Use of orlistat in the community

Fig 5

Total use of Alzheimer's drugs in the community