The health and care system in the UK looks after an ageing population with increasingly complex care needs, but money is tight. This means the NHS needs to:
- develop new ways of providing care and,
- when a new approach works well, understand and share learning about that success with other parts of the health and care system quickly.
To meet this challenge, the Rapid Service Evaluation Team (‘RSET’), comprising health service researchers, health economists and other colleagues from University College London and the Nuffield Trust, have come together to rapidly evaluate new ways of providing and organising care.
We have been funded by the National Institute for Health Research (NIHR) Health Service and Delivery Research (HS&DR) programme for five years, starting on April 1st 2018. The NIHR HS&DR programme have also funded a second team, the Birmingham, RAND and Cambridge Evaluation (‘BRACE’) Centre, which is a collaboration between the University of Birmingham, RAND Europe, the University of Cambridge and National Voices. The two teams will coordinate their evaluation programmes to maximise benefits from our work.
What are we doing?
We are studying the latest changes in health and care services, and provide meaningful lessons about these changes in a timely manner.
We are looking at changes in terms of:
- the impact of services on how well patients do (e.g. their quality of life, how likely patients are to recover)
- whether services give people the right care at the right time
- whether these services are good value for money
- how changes are put into practice, and what patients, carers, and staff think about how the changes happened and whether they think the changes made a difference
- what lessons there are for the rest of the NHS and care.
What are we evaluating?
We are evaluating service innovations – those driven both by national policy, and local needs – from innovative organisational hospital forms such as hospital groups and innovations in the management of services to new ways of delivering services to patients, and quality improvement initiatives.
We are working with our Stakeholder Advisory Board (a specially selected group of national experts, including patient and public representatives from a wide range of settings) and others to identify health and care innovations to evaluate.
How are we evaluating?
Different service innovations will require different approaches to their evaluation. We are using a range of quantitative and qualitative approaches as appropriate to each evaluation, including, analyses of administrative and other datasets, economic analysis, stakeholder interviews and focus groups, surveys and discrete choice experiments.
How are we sharing results?
We want the findings from our research to have a timely impact. We believe that sharing findings as they emerge will increase their impact and benefit for patients, the public, and the health and care systems. Therefore we share our findings rapidly with the people who are making changes.
For each project, we are working with our partners to agree how best to share lessons from our work effectively. We are experts in doing this, and have excellent links with health and care services across the country.
We are also making use of user-friendly summaries, blogs and webpages, workshops, present at conferences, and publish research articles that can be read freely by the public.
Who do we work with?
We involve patients, and the public, clinicians, and those running health and care services at every stage of our research, both within projects and across the programme as a whole.
We work closely with the NIHR, and be guided by our Stakeholder Advisory Board, which looks at
- whether we are doing what we set out to do
- keep us up to date on the latest service changes, and
- tell us what they think about our new plans for research.
If you have an idea for a service innovation we might evaluate, or want to get in touch, please contact us at RSET@nuffieldtrust.org.uk
Evaluations in progress
- Special measures for quality improvement and challenged providers: evaluating the impact of improvement interventions in NHS trusts
Evaluations in the pipeline
- Innovations in outpatient services
- Youth violence intervention programme for vulnerable young people attending Emergency Departments
- Centralisation of specialised services
Previous evaluations from Nuffield Trust and UCL
Special measures for quality improvement and challenged providers: evaluating the impact of improvement interventions in NHS trusts
Status: in progress
The Special Measures for Quality regime (SMQ) provides NHS trusts with increased support and oversight from NHS Improvement (NHSI) following a Care Quality Commission (CQC) inspection that has rated as inadequate the leadership and one other domain. There is also a ‘watch list’ of challenged NHS providers at risk of going into SMQ that receive support (this list is not in the public domain). According to NHSI, trusts in Special Measures for Quality and those on the ‘challenged providers’ list can receive a combination of three main interventions to tackle quality problems: 1) Improvement Director, 2) buddying, and 3) access to central funding. These might be delivered in conjunction with other interventions and is often within a context of leadership changes at board level.
There is currently limited knowledge about whether, and how, targeted interventions from NHS Improvement drive improvements in quality, how much they cost, and whether the current improvement regime strikes the right balance between support and scrutiny. There is also interest in understanding why some trusts become sufficiently resilient and avoid re-entering special measures or exit the regime sooner than others.
Therefore, this study – to be completed by December 2019 - will evaluate the responses of NHS providers to the SMQ and challenged providers’ regimes. The aim is to determine how trusts respond to the various SMQ interventions and how they may impact on organisations' capacity to achieve and sustain quality improvements over time.
Following a scoping exercise which included the types and characteristics of trusts subject to the special measures regime since 2013, as well as a survey of the literature to identify key themes and concepts relevant to this evaluation, a set of research questions and design methods have been developed.
Potential outputs from this evaluation will include:
- Lessons for trusts on responding to interventions in SMQ/challenged providers regimes, such as what to prioritise and the organisational capabilities found to support quality improvement
- Lessons for the Department of Health and Social Care and NHSI on how to support challenged providers and those in the SMQ regime in light of the varied internal and external contexts of health care organisations facing performance difficulties
- Recommendations on the use of routine, quantitative data to track quality improvements, including at system level, and the potential costs and benefits of interventions of the SMQ regime
- A conceptual framework to help evaluate the SMQ regime and challenged providers programme in future.
Project lead: Professor Naomi Fulop
Innovations in outpatient services
Status: in the pipeline
The NHS Long Term Plan set an ambitious aim of reducing the number of outpatient attendances by 30 million over the next ten years. In part this is a reaction to the huge growth in appointments and attendances over the last decade and in part to possibly better ways of delivering care, reducing inefficiencies and improving patient experience and outcomes.
Initial quantitative work will attempt to identify trusts/specialties that appear to have successfully slowed, halted or reversed the rise in outpatient appointments observed nationally. In addition to outpatient volumes, we will also identify trusts with atypical positive trends in DNAs/cancellations and in tele consultations. Further contextual analysis would be carried out to identify, for example, whether there are changes to population characteristics that might explain any changes observed.
Further qualitative methods aiming to investigate whether specific trends identified might be attributable to particular innovations in outpatient services.
A second phase of the work will aim to robustly evaluate, with mixed methods approaches, a limited number of innovations. These would be selected from a shortlist of services identified during the initial phase, or from other innovations identified by RSET.
Youth violence intervention programme for vulnerable young people attending Emergency Departments
Status: in the pipeline
Redthread – a charity set up in 1995 - embed crisis-intervention specialist youth workers into existing health systems, to capitalise on ‘teachable moments’ in order to engage young people and encourage positive change in their lives.
An important aspect of any evaluation of Redthread interventions is the range of possible outcome measures to consider - from those related to health and health care, to education, social services, employment and the criminal justice system.
The scoping phase for this evaluation aims to:
- Review previous evaluations to identify research gaps and methods employed
- Review the current evidence base on interventions in frontline emergency or trauma services aimed at reducing violence and/or targeted at young people
- Assess the feasibility of a prospective evaluation at one London trust
- Assess the possibility of retrospective evaluation at one or more of the ongoing sites
- Investigate data collection/linkage issues concerning multiple health and non-health outcome measures
Centralisation of specialised services
Status: in the pipeline
There have been significant changes in the provision of specialist clinical care in the NHS in recent years, with plans to centralise specialist services for some conditions (e.g., acute stroke, major trauma, specialist cancer surgery, cardiac surgery, vascular surgery, specialist paediatrics services) into fewer centres. Greater concentration of specialist health care services into fewer centres was identified in both the Five Year Forward View and NHS Long Term Plan as a future model of care that should be considered within the NHS.
One rationale for centralisation is that it leads to increased volumes of patients being treated at specialist centres. This means that more patients have access to specialist staff expertise and innovative technologies and treatments. Also, a higher volume of patients means that staff obtain greater experience and expertise in dealing with cases. On the downside, centralisation is likely to increase distances travelled to hospital and travel times, increasing the time and money costs to families of receiving care.
The aims of this study are to use quantitative and qualitative research methods to investigate the underlying dimensions of centralising specialist health care services in the UK, and investigate whether and how patients, general population, health care professionals, hospital managers and commissioners would prefer them to be centralised.
The objectives are:
1. To undertake a scoping review to identify what “centralisation” as a service innovation means in the context of specialist health care services, and what the dimensions of centralisation are.
2. To conduct a discrete choice experiment (DCE) to analyse preferences for centralisation of specialist health care services by patients, general population, health care professionals, hospital managers and commissioners.
3. To develop a taxonomy to map the different models of centralisation reported so far in the literature and use this information to inform the sampling of the centralisation case studies.
Other blogs on rapid evaluation
Read Arne Wolters' take on 'Rethinking evaluation: the rigour of rapidity' on the Health Foundation website