Fast tracked: the vital importance of rapid evaluation

As health and care systems have been grappling with the pressures of a global pandemic while also caring for a growing and ageing population, a quiet revolution has been taking place in health services research.

Rapid evaluation – that is, the robust evaluation of new services taking place at speed and in collaboration with the people delivering these services – has been around for some time, but in recent years has become an essential tool in the innovator and policymaker’s toolbox. And our work for the Rapid Service Evaluation Team funded by NIHR has been at the cutting edge, bringing fresh insight to health and care service models developed at pace, informing their development and roll-out across the NHS.

As we announce that we have received funding to commence another five years of evaluating vital innovations in health and care, it’s worth pausing to reflect on what the impact has been so far of this work, what the future looks like for the use of rapid evaluation methods in understanding health and care, and why the pandemic and continued pressures on the health system mean it has never been more relevant.

Doing things differently

Back in 2018, when the world was peacefully unaware of what was about to follow, the NIHR commissioned two rapid evaluation teams: NIHR RSET, a collaboration between University College London, the Nuffield Trust and the University of Cambridge, and NIHR BRACE run out of the University of Birmingham, to carry out a five-year research programme bringing rapid methods to evaluating health and care innovations.

At the time, nobody could have foreseen the pressure that the health and care systems would be under as the pandemic spread, forcing health and care providers and commissioners to adopt new services and models at speed and scale across the UK. And as health and care services adapted to this new world, our researchers stepped up to the challenge, working collaboratively with many of them to understand the impact that changes in services were having on the delivery of health care.

The exchange of formative feedback and regular dialogue between researchers and the people delivering services is of vital importance in all evaluations (not just those that are rapid). But in rapid evaluations, researchers don’t spend years collecting data and writing up their research before launching it onto an unsuspecting world, which may have changed immeasurably since the research was started. Instead, they work with service providers, decision-makers and innovators, bringing research insight into the development of services and how they are adopted. However, such collaborative working presents particular challenges in rapid research, as building relationships quickly can be difficult. Researchers working on rapid evaluations need to find ways of building trust quickly.

Methods include short-study timeframes, participative approaches, team-based research and iterative data collection and analysis. The idea is that this approach means research has more real-world impact, informing and supporting the development of on-the-ground policymaking rather than existing in the realms of academic and public debate. As Professor Chris Whitty presciently argued in 2015: “An 80% right paper before a policy decision is made it is worth ten 95% right papers afterwards, provided the methodological limitations imposed by doing it fast are made clear.”

Timely impact

And the impact has been significant. We led a very rapid study of remote home monitoring services with pulse oximeters of Covid-19 patients, developed during the first wave of the pandemic, to inform the national roll-out of these services during winter 2020/21. We then evaluated this national roll-out. Findings from this study have fed through into wider policy on the use of virtual wards in the NHS.

We also evaluated a scheme where paramedics used a tablet to video call with stroke specialists to assess whether a patient is having a stroke, with the aim of helping patients get to the right services first time. We learned that this made a big difference to patient journeys, with many non-stroke patients avoiding transfers to a specialist stroke unit. Importantly, we found that paramedics and stroke specialists liked this innovation and felt it was safe for patients. Our work informed the decision to launch a national pilot, which is testing this approach in every region of England, with the potential to benefit thousands of stroke patients in these areas.

Other evaluations have informed the development of youth violence reduction initiatives in hospital, the use of special measures regimes, and are currently influencing NHS England’s roll-out of patient-initiated follow-up in better managing outpatient appointments.

A vital part of the future

So what next for rapid evaluation? With the worst of the pandemic apparently behind us, is there still a need for such methods to bring researchers and service designers together in real-time? The answer is a resounding yes: the challenges facing health and care services now are, if anything, even more sobering than back in 2020, with a burnt-out and overstretched workforce, an even bigger backlog of treatments and operations, and the challenge of delivering more for less across public services. These challenges, alongside the development of new technologies and opportunities for service change, have driven huge demand for rapid evaluation and are why the NIHR have built on the work we and BRACE did, and are now funding five separate units to carry out rapid evaluations in health and care, of which we are one.

Our approach will continue to draw on the methods we have developed over the past five years and bring these to shedding light on how effective innovations are across the NHS. We are already getting stuck in, looking at the implementation of AI in imaging and whether it is safe to utilise ready-to-use intravenous medicine on inpatient wards. Like the first phase of RSET, we will be publishing our work as we go along, sharing insights with those involved in service design and bringing our wider learning to the health and care policy community.

With health and care having to constantly innovate to adapt to change – whether that change is a global pandemic or the advent of artificial intelligence – rapid evaluation is an approach whose time has come.