The health and care system in the UK looks after an ageing population with increasingly complex care needs, but money is tight. This means the NHS needs to:
- develop new ways of providing care and,
- when a new approach works well, understand and share learning about that success with other parts of the health and care system quickly.
To meet this challenge, the Rapid Service Evaluation Team (‘RSET’), comprising health service researchers, health economists and other colleagues from University College London and the Nuffield Trust, have come together to rapidly evaluate new ways of providing and organising care.
We have been funded by the National Institute for Health Research (NIHR) Health Service and Delivery Research (HS&DR) programme for five years, starting on April 1st 2018. The NIHR HS&DR programme have also funded a second team, the Birmingham, RAND and Cambridge Evaluation (‘BRACE’) Centre, which is a collaboration between the University of Birmingham, RAND Europe, the University of Cambridge and National Voices. The two teams will coordinate their evaluation programmes to maximise benefits from our work.
What are we doing?
We are studying the latest changes in health and care services, and provide meaningful lessons about these changes in a timely manner.
We are looking at changes in terms of:
- the impact of services on how well patients do (e.g. their quality of life, how likely patients are to recover)
- whether services give people the right care at the right time
- whether these services are good value for money
- how changes are put into practice, and what patients, carers, and staff think about how the changes happened and whether they think the changes made a difference
- what lessons there are for the rest of the NHS and care.
What are we evaluating?
We are evaluating service innovations – those driven both by national policy, and local needs – from innovative organisational hospital forms such as hospital groups and innovations in the management of services to new ways of delivering services to patients, and quality improvement initiatives.
We are working with our Stakeholder Advisory Board (a specially selected group of national experts, including patient and public representatives from a wide range of settings) and others to identify health and care innovations to evaluate.
How are we evaluating?
Different service innovations will require different approaches to their evaluation. We are using a range of quantitative and qualitative approaches as appropriate to each evaluation, including, analyses of administrative and other datasets, economic analysis, stakeholder interviews and focus groups, surveys and discrete choice experiments.
How are we sharing results?
We want the findings from our research to have a timely impact. We believe that sharing findings as they emerge will increase their impact and benefit for patients, the public, and the health and care systems. Therefore we share our findings rapidly with the people who are making changes.
For each project, we are working with our partners to agree how best to share lessons from our work effectively. We are experts in doing this, and have excellent links with health and care services across the country.
We are also making use of user-friendly summaries, blogs and webpages, workshops, present at conferences, and publish research articles that can be read freely by the public.
Who do we work with?
We involve patients, and the public, clinicians, and those running health and care services at every stage of our research, both within projects and across the programme as a whole.
We work closely with the NIHR, and be guided by our Stakeholder Advisory Board, which looks at
- whether we are doing what we set out to do
- keep us up to date on the latest service changes, and
- tell us what they think about our new plans for research.
If you have an idea for a service innovation we might evaluate, or want to get in touch, please contact us at RSET@nuffieldtrust.org.uk
Evaluations in progress
1. Special measures for quality improvement and challenged providers: evaluating the impact of improvement interventions in NHS trusts
The Special Measures for Quality regime (SMQ) provides NHS trusts with increased support and oversight from NHS Improvement (NHSI) following a Care Quality Commission (CQC) inspection that has rated as inadequate the leadership and one other domain. There is also a ‘watch list’ of challenged NHS providers at risk of going into SMQ that receive support (this list is not in the public domain). According to NHSI, trusts in Special Measures for Quality and those on the ‘challenged providers’ list can receive a combination of three main interventions to tackle quality problems: 1) Improvement Director, 2) buddying, and 3) access to central funding. These might be delivered in conjunction with other interventions and is often within a context of leadership changes at board level.
There is currently limited knowledge about whether, and how, targeted interventions from NHS Improvement drive improvements in quality, how much they cost, and whether the current improvement regime strikes the right balance between support and scrutiny. There is also interest in understanding why some trusts become sufficiently resilient and avoid re-entering special measures or exit the regime sooner than others.
Therefore, this study – to be completed by December 2019 - will evaluate the responses of NHS providers to the SMQ and challenged providers’ regimes. The aim is to determine how trusts respond to the various SMQ interventions and how they may impact on organisations' capacity to achieve and sustain quality improvements over time.
Following a scoping exercise which included the types and characteristics of trusts subject to the special measures regime since 2013, as well as a survey of the literature to identify key themes and concepts relevant to this evaluation, a set of research questions and design methods have been developed.
Potential outputs from this evaluation will include:
- Lessons for trusts on responding to interventions in SMQ/challenged providers regimes, such as what to prioritise and the organisational capabilities found to support quality improvement
- Lessons for the Department of Health and Social Care and NHSI on how to support challenged providers and those in the SMQ regime in light of the varied internal and external contexts of health care organisations facing performance difficulties
- Recommendations on the use of routine, quantitative data to track quality improvements, including at system level, and the potential costs and benefits of interventions of the SMQ regime
- A conceptual framework to help evaluate the SMQ regime and challenged providers programme in future.
Full study protocol for this evaluation.
Project lead: Professor Naomi Fulop
Previous evaluations from Nuffield Trust and UCL
Lessons from stroke reconfiguration in London and Greater Manchester
Spotlight • A collection of the Nuffield Trust's work on evaluation
Other blogs on rapid evaluation
Read Arne Wolters' take on 'Rethinking evaluation: the rigour of rapidity' on the Health Foundation website
Project co-directors
Chief EconomistNuffield Trust
Professor of Health Care Organisation & ManagementUCL
Professor of Health EconomicsUCL
Project team
Senior Research AnalystNuffield Trust
Research AssociateUCL
Senior Research AssociateUCL
Senior Research AnalystNuffield Trust
Research AnalystNuffield Trust
Research Project ManagerUCL
Patient and Public Involvement LeadBarts Health NHS Trust
Research AssociateUCL
Blog post