Doing right when you are wrong: perspectives on workforce planning in the NHS in uncertain times

The health service’s most important asset is its staff. Ahead of a promised national workforce strategy for the NHS, Billy Palmer takes a closer look at workforce planning in the health service over the years, describes the challenges involved in getting it right, and outlines what a successful workforce plan should cover.

Long read

Published: 13/06/2022

You wait for ages for one and then two come along. And so may it be with NHS workforce strategies, with a new long-term strategic framework impending and a “proper long-term workforce plan” promised to follow.

Much is hoped for these documents. In fact, tabled amendments to the Health and Care Bill around strengthening workforce planning, including having independent staffing projections, were rejected on the basis of these upcoming publication. But does the NHS have the right approach to making such staffing projections and developing workforce plans around them?

A chequered history?

The focus on workforce planning is obviously welcome. But the best-laid plans of mice and men often go awry. And, indeed, the past is scattered with previous, failed projections which went on to underpin subsequently flawed workforce plans. A decade ago, projections – admirably based on a bottom-up workforce planning exercise but influenced by undeliverable financial plans – proved wildly inaccurate almost immediately afterwards. In fact, within two years, they had underestimated the demand for staff by around 24,000 nurses. To compound matters further, there was also a failure on the other side of the equation: supply. Specifically, the number of adult nurses in 2018 fell below even the ‘worse case’ scenario made three years prior in the national workforce plans.

While there are also more recent examples of shaky projections and flawed workforce plans – GPs being a current one – such failures are, in fact, pretty much as old as the NHS itself. In 1955, a committee of the “great and good” determined that there was a risk of training too many doctors and recommended reducing student intakes moving forward. They came to the wrong answer in part due to underappreciation of numbers emigrating and immigrating. The Chair’s name, (Sir Henry) Willink, subsequently became a byword for disastrous planning at the time. That said, he was not alone: five years earlier the BMJ had suggested that the current size of the medical profession was satisfactory and did not need expanding.

Why projections can fail?

Projections of supply and demand can of course help better estimate likely future patient needs as well as the numbers joining and leaving the workforce. In fact, this can overcome the failures of recent history, whereby a pervasive optimism meant forecasts tended to overestimate the number of staff available, while underestimating the number that would be needed – partly due to linking workforce plans to undeliverable financial plans.

But they will be wrong, to a degree. There are inherent reasons for this. Key among them are that the health service’s demand for staff is, in fact, primarily determined by a combination of the level of funding available and the cost of employing staff. On the latter, pay settlement outcomes are somewhat unpredictable while the former – funding – is a largely political decision that has proved difficult to estimate and often changes even within a given financial year.

And, on politics, even where there are evidence-based projections on the future size and skill-mix needed, this may be trumped by manifesto and similar targets, given the appeal of announcing tangible goals around extra nurses or doctors. Also, the responsibility for this uncertainty around demand for staff doesn’t just lay at the door of politicians. For example, the report on failings in care at Mid Staffordshire NHS Foundation Trust and the safe staffing guidelines for adult nurses coincided with a spike in demand for staff. The subsequent higher requests for temporary staffing to fill posts equated to an estimated additional £230 million annual cost to the NHS.

There is also a large degree of uncertainty in predicting the supply of staff available too. For example, the NHS’s ability to bring in and retain staff can be affected by other countries’ overseas recruitment strategies given the global nature of the clinical labour market. One such instance is the recruitment of more than 46,000 overseas nurses by the USA in just two years in the mid-2000s, which is likely to have affected other countries’ ability to bring in staff from abroad. Similarly, regulations like the introduction of language tests for all overseas nurses can affect supply.

Moreover, as we are all too acutely aware, there can be fundamental shifts – such as in the case of the pandemic – which dramatically alter both supply and demand in a way that could not be captured accurately in a prior workforce projection. These uncertainties are compounded by some known-unknowns. While there is some reasonable information on activity and staffing within the hospital sector, less is known about general practice and the independent sector as well as other sectors where staff may join from or leave to, including social care.

Why can’t we secure the staff needed?

Aside from the questions about their ability to accurately predict, forecasts – in themselves – do not secure staff and cannot address imbalances in supply and demand. In this respect, workforce projections may be necessary but certainly are not sufficient. But developing a strategy to minimise – proactively and reactively – both the extent of the staffing imbalance and any undesirable effects carries its own inherent challenges.

First, there might not be consensus on what is the most cost-effective strategy. Determining this involves complex decisions given the various mechanisms available (such as temporary staffing, overseas recruitment, and measures around domestic training and retention of existing staff), each of which carry their own costs and capacity to address imbalances. Also, overall value for money of different levers is not obviously apparent – in part as those employing or training the staff are often only partially exposed to the true costs and benefits (we noted this recently with respect to overseas recruitment and domestic training).

Second, there is a trade-off between meeting short-term demand and sustainability in the long term. For example, the number of doctors-in-training needed to deliver services now might not match future demand for them as qualified consultants, with that balancing act skewed by the fact that NHS providers are only having to meet half the basic pay costs of those in training (with the remainder paid through national training budgets), which has the effect of effectively making them a subsidised resource. There is also a risk of so-called present bias whereby short-term benefits (such as in-year affordability by restricting training posts) are prioritised over long-term benefits, such as supply of clinical graduates in the future. The time horizons of some – including senior leadership who are held to account for immediate performance and politicians – may be particularly short.

Third, the risks of over- and under-supply need to be balanced appropriately. Historically the NHS has prioritised the importance of avoiding over-supply – perhaps anxious of potential press about unemployed medical graduates trained at great personal and taxpayer expense, even though the costs of an under-supply are higher from virtually every perspective. An added complexity is that the risks of, and ability to mitigate, staffing imbalances differ by profession, with changing skill-mix having the potential to remediate in some areas, but which isn’t possible where a shortage of, say, neuro-surgeons occurs.    

Fourth, there are some financial, organisational and contractual barriers and opportunities associated with seeking to change the supply and retention of staff. For example, where training is centrally funded, any expansion in one area may well require – given finite budgets – some professions seeing cuts in their pipeline. The higher-education institutions that deliver courses and NHS providers that host clinical placements may also only be able to expand enrolment by a limited amount from year to year given the capacity and support it requires.

On an operational point, there are also key considerations about who and when actions should be taken. This is particularly important, and somewhat unresolved, given emerging responsibilities of integrated care systems at a local level. Historically, NHS workforce planning responsibilities have been recognised as “too disjointed at a national and local level”. Certainly, there are some areas which need coordination given, for example, the upfront costs and administration of overseas recruitment potentially unworkable for some services or settings without support.

A brighter future?

It is right that, moving forward, policy-makers and researchers seek to improve workforce projections, but the endeavour must not end there. Given such workforce trends carry much uncertainty, it is not a case of whether there will be a mismatch between the demand for (and supply of) staff, but rather the direction and size of it. From this perspective, the challenge is about how quickly and affordably current and likely future staffing imbalances can be addressed. With this in mind, much relies on the “proper” workforce plan promised by the Secretary of State, which should cover off the following as a minimum to be successful:

  1. Detail on the capacity and costs of the various mechanisms available to address different short- or possible long-term staffing imbalances, covering both supply and retention. This should enable decision-makers to deliver the most cost-effective workforce solutions from patient and taxpayer perspectives.
  2. Clear articulation of the governance arrangement, with appropriate alignment of roles, responsibilities and funding to incentivise optimal value-for-money in delivering the workforce strategy. This should include – but not be limited to – health, social care and education sectors and cover national, regional and local organisations.
  3. An assessment of how it can be successfully delivered by all services, in all parts of the country. The strategy needs to reflect every region, setting and provider, so that no areas or services are unduly exposed to workforce challenges.
  4. A plan to ensure there are sufficiently and appropriately flexible training pathways, to ensure that the skillset of those completing courses will be able to meet the needs of the NHS cost effectively in the short and long term.
  5. An evidence-based view on balancing national and local risks of over- and under-supply. This should also consider extreme but plausible unforeseen changes to the demand or supply of staff and the strategic response to these (such as creating temporary registers and use of students), and acknowledge the value to the wider public sector and to the graduate of various clinical qualifications.
  6. Reflection of how other health systems and public services manage similar challenges on workforce planning.
  7. An assessment of how the funding arrangements and current contractual terms for the supply of clinical staff are affecting the ability to change the supply pipeline, skill-mix, and the balance between addressing current and likely future imbalances. This includes, but is not limited to, the national subsidising of NHS providers’ pay costs for doctors-in-training and contracts with higher-education institutions or clinical placements providers. 

The NHS’s most important asset is – and will continue to be – its staff. With a workforce rife with burnout from efforts over the pandemic and the prospect of a step-change in activity to attempt to make headway into the huge waiting lists, managing the NHS workforce has never been as important. Efforts to do so must couple any workforce forecasts with an immediately deliverable strategy about how to react to ever-evolving imbalances between supply and demand. And if the forthcoming national workforce strategy does not do so then it will have likely missed the point.