In an effort to improve the quality of health care and reduce the financial pressure on the NHS, efforts are being made to deliver more care in community settings, with the aim of preventing unnecessary and expensive admissions to hospital. We are developing methods for evaluating how well these interventions perform.
Recent developments in data linkage mean that it is now possible to provide evidence on the impact of innovations on secondary care utilisation in a timely and less resource-intensive manner.
The Nuffield Trust has developed evaluation methodologies that exploit the large amount of administrative information on individual patients that is available in the NHS and social care.
We have used these methods to evaluate more than 30 community-based service innovations in the NHS over the past five years. Our focus has usually been on whether the service model being implemented and evaluated has had an impact on service use and costs, using quantitative methods.
We believe the paper will provide useful learning for the new health and social care integration ‘pioneer’ sites that will be appointed by the Department of Health by September 2013Many preventive interventions are available at the NHS’ disposal, ranging from case management for high-risk patients, the use of technology such as telehealth and telecare to monitor patients remotely, and changes to reimbursement systems for care provided by hospitals.
Evaluation methods: in detail
One particularly robust way of evaluating interventions is by conducting a large randomised controlled trial, where one group receives the new intervention and a control group receives usual care, such as our project that is evaluating the impact of telecare and telehealth.
This project is exploring new, less expensive, ways of evaluating how effectively new preventative interventions in the NHS are working
However, trials like this are not always a feasible option for the NHS or social care as they require a substantial investment of money and time. This project is examining alternative options that make the most of the large amount of administrative information on individual patients that is available in the NHS and social care.
In this programme of work we are applying person-based risk-adjusted evaluation (which allows us to put together a control group for comparative purposes by linking up information stored on patients’ care) to a number of interventions in health and social care, including selected Partnership for Older People Projects. We are also conducting more research into the statistical methods used to select control groups which evaluate the success of interventions.
The key to finding a control group lies in using the large amounts of patient information held throughout the health and social care system, which can now be extracted for each patient and linked together anonymously without compromising their confidentiality. Since we are talking about information on hundreds of thousands of individuals, there is often scope to find controls that were sufficiently similar to those who received the service being evaluated but who did not actually receive it themselves.
This means that we can now match an individual receiving the new service to a control group that were similar in terms of age, sex, recorded health diagnoses and prior hospital use, and who lived in an area with a similar level of deprivation. In the past this would have been very challenging.
This kind of evaluation is economical and can be applied retrospectively, so that pilots can be evaluated even if at first they were not developed with evaluation in mind. The method can be applied in real time too, so it can be used to refine a service in response to emerging evaluation findings. However, controls must be selected carefully because, unlike a randomised controlled trial, matching ensures only that the two groups are similar in ways that can be observed.
This page will pull together developments from our research project to develop evaluation methods further, including published papers and details of events.