Recently, concerns have been raised over the sufficiency, distribution and sustainability of New Zealand’s medical workforce, with competing views being offered on the responses that should be taken.1,2 The arguments presented for and against proposed solutions tend to be medically orientated and supported by evidence and the analysis of trends that are framed by the present health system’s organisation. This has the outcome of further embedding the present’s infrastructures and service delivery methods, which limit alternative consideration of how future services could be provided and peopled.3 In the main, suggested solutions seek to address New Zealand’s medical workforce imbalances by reducing reliance on international medical graduates (IMGs), to increase the numbers and placements of locally trained doctors through medical school roll increases, and by continuing or improving a range of incentives and informational programmes as a means to promote and attract trainees for hard-to-staff vocational specialities.4–6

Yet despite numerous similar policy interventions over past decades, the issues of persistent shortages and misdistributions continue.2 This is in part due to a traditional medically focused and a by-profession approach to health workforce policy and planning, as well as a reliance on quantitative forecasts made under conditions of uncertainty.7 In New Zealand, these forecasts, made available by academics and professional organisations,8,9 with regularly collected workforce survey data are also used as an aid to project the future numbers of particular professions.10,11 However, this approach tends to neglect the fact that health workforce planning involves a range of constituents, some of whom exhibit various states of rivalry and tension. These conflicts, which can impede health workforce plans,12 are found across the health sector between constituents such as politicians, managers and clinicians and within institutions, and may be expressed as contradictions in authority and accountability between management and professionals, and as tensions between continuing business-as-usual and introducing innovation.13 As such, health workforce planning struggles to achieve ‘right or optimal’ solutions, while the needs of its multiple constituencies may or may not be met by the policy or policies that are chosen.14

Hence, in this article, we offer some insights provided by a recent study of health workforce planning in New Zealand.15 We begin by discussing the limitations of traditional health workforce planning. We then move on to introduce types of approaches considered to be more appropriate for studying the future under conditions of increasing uncertainty. We conclude with an overview and suggest a rethinking of the practices and policy frames used to plan for future health workforces.

The planning of health workforces

The aim of health workforce planning is to determine the balance between workforce demand and supply that ensures the availability of sufficient appropriately qualified personnel.16 Due to considerable differences between health systems’ designs and operations, a range of models and methods are applied to achieve this task.16,17 In the main, these models quantitatively estimate expected service demand. These estimates are then used to approximate future workforce supply and adequate numbers of professionals or specialist roles, to determine workforce numbers sufficient to meet future health needs or to realise benchmarked population-provider ratios.18 There is a range of limitations to these health workforce forecasting approaches.19

Predictive approaches to forecasting tend to be best suited for situations that are able to be expansively mapped, measured and modelled.20 As such, they tend to perform well enough in situations where there are few surprises, for example short-run economic or financial decisions.20, 21 Over longer time frames their accuracy diminishes.18 Predicting outcomes in social systems is more difficult due to these systems’ variable relationships, discontinuities and interactions along with the effects of time.21 In social system situations, predictive forecasts are not failing due to poor execution, but rather because they are trying to do the impossible, dealing as they are with the system’s complexity and contradictory interdependencies, and through minimising the effects of human agency by assuming that behavioural variables remain constant.20 In response, there has been a range of efforts to reduce workforce forecast inaccuracy by improving the models, by taking more systematic approaches17 and by rethinking.7 The latter seeks to look past the numbers and use clinically-led visions of future services to build an understanding of how to achieve the desired outcome of meeting future demand with an appropriate and financially viable workforce.7,22

Rethinking also requires that the problem to be addressed is understood.23 Some problems are relatively easy to define and describe, and therefore straightforward to solve. Others suffer from lack of information that would enable a solution to be easily found, while yet others, such as wicked problems, express irreducible complexity.21 Popularised by Rittel and Weber,24 a wicked problem cannot be described as objectively correct or false and nor does it have a definitive solution. It is also exacerbated by poor forecasting conceptualisation, insufficient data or intelligence and a plurality of constituent objectives, resulting in contradictions where a solution for one constituent generates a problem for another.24 Health workforce planning, like a number of other health management issues, exhibits the wicked problem’s features of having no ‘right or optimal’ solution, poor forecasting outcomes and constituent diversity.25,26

This is problematic as, comparable to other areas of public policy, health workforce planning is also dominated by a culture of predictive data and evidence-based decision-making.27 When this culture is coupled with an institutional intolerance of uncertainty, it leads to the overuse and over-reliance on forecasts leading to projections being treated as facts or what Veenman28 termed certainification, or a strengthening of the illusion of prediction reliability.29 Faced with poor results, policymakers are likely to refine and improve on these techniques, further embedding error and forecast inaccuracies.30 In addition, policymakers tend to be under political pressure to produce more accurate predictions, to use better models and, in particular, to avoid mistakes.31 This in turn reinforces policy makers’ propensity to dismiss assistance from non-numeric forecasting.27

Forecasting under conditions of uncertainty

The range of forecasting’s non-statistical techniques are termed judgemental and can be characterised by those that predict one’s own future behaviour or the future behaviours of others determined by experts.32 Judgemental methods may also integrate aspects of statistical methods to quantify proposed relationships, producing semi-quantitative forecasts by providing probabilities or weights to opinions.33 Judgemental forecasting shares a number of methods with another discipline of futures studies termed ‘foresight’32,33 (see Figure 1). While forecasts produce logical results as predictions or projections based on the past, foresight copes better with conditions of uncertainty and expresses the future as a range of possibilities28 (see Figure 2). Thus, for longer-term views of the future, techniques shared by forecasting and foresight such as scenarios, Delphi and intentions, elucidated through stakeholder or actor surveys or interviews, are able to provide judgmental narrative and semi-quantitative data that can be used as part of describing the problem or to design more representative short-run models or simulations.15

Figure 1: Intersection between forecasting and foresight methods.

Figure 2: Forecasting, foresight and futures studies.

‍

A potential benefit of using judgement-based techniques in the context of health workforce planning is that foresight’s strength rests with its ability to account for the “difficult-to-quantify, dynamic properties of any given set of complex circumstances, particularly where social systems and markets are concerned”.34 Scenarios, particularly those developed with stakeholder input, have been found to be useful for addressing wicked problems, enabling the causes of future outcomes to be understood and to allow the problem’s dimensions to be examined more fully.23,35 The use of scenarios has also been shown to alter people’s expectations of future events, which is particularly useful should the constituents be rivals, as scenarios allow constituents to project themselves into future situations or to ask them to explain the depicted outcomes, particularly should the scenario storylines be inconsistent with a constituent’s preconceived biases.36

Where are we in New Zealand?

New Zealand produces scenario-based data as part of its workforce planning processes, through Health Workforce New Zealand’s Work Service Reviews (WSRs). Described as sets of possible future clinical scenarios for service aggregates and generated by clinical subject matter experts and opinion leaders, the WSRs are planning analyses that, along with cross-sectional analyses of by-profession or role-based workforce forecasts, aim to provide robust planning intelligence. 7 In the health workforce study referred to earlier in this article,15 WSR data and clinical-derived service preferences were aggregated by health sub-sector and re-analysed. This re-analysis revealed that many of the solutions and policies being offered by present health workforce interventions are aimed at fixing worker stocks and flows, or maintaining numbers. However, these interventions are at odds with the approach that Meadows considers to be effective for leveraging system change, which involves a focus on system goals and its organisation, rather than managing numbers.37 The numbers-based policy interventions that we found here in New Zealand tended to be representative of and promoted the present’s health workforce organisation and were mostly reactive to immediate presenting issues. More encouraging, though, is the finding that New Zealand’s workforce constituents seem to be in general agreement that a system’s funding structures and models of care are most important in terms of determining future workforces. Exploring these fundamental levels of a system’s organisation, rules and goals is, as Meadows points out, more effective in terms of leveraging system change.37

By taking the aggregated WSR themes, a number of scenario storylines were able to be constructed, providing a normative or common vision of how the future ought to be, along with a few relevant alternatives. Opinions of these scenarios’ desirability and likelihood were collected using two expert sector panels, which provided detailed insight using the managed polling technique of policy Delphi. Here the aim was to gather the widest range of opinions, rather than seeking agreement or consensus, as a means to identify the areas of enablement and those policy areas that may lead to conflict. These panels revealed that the organisation of the health sector and the mechanisms through which services are delivered present as much of a barrier for workforce sufficiency, as does the present reliance on parameter-level interventions to produce, retain and maintain the workforce. This observation starts to point to why many of New Zealand’s health workforce problems persist, and also provides some insight into why the range of corrective approaches and pilot projects have not diffused further across the health system. For instance, the panels indicated that the current models of care, funding parameters and sector organisation patterns in primary care disincentivise the introduction of new roles, care options and, in particular, the involvement of patients to be part of the design and introduction of new service configurations. Importantly, the panels saw that services should be collectively led with a strong clinical presence, though pointed out that current service delivery models may also be a contributor to the shortages of some roles. A health service’s business model as well as its model of care are two factors that are identified as impediments to the advancement of role innovation and skill-mix adoption or for a diffusion of workforce roles that would be useful for future integrated care formats.38 Thus a consideration of future workforces, their roles, numbers and avenues of training is required when considering service configurations, models of care and funding decisions at national, regional and local levels. Further, business models that permit the use of specialised community-based roles and enable access to further training and qualifications will act to incentivise and motivate current staff to upskill and take on these more specialised care and treatment tasks. These business models have implications for planning and ensuring a fit for purpose future workforce, particularly when New Zealand’s vision is of a more integrated and patient-centred system of health delivery.39

Conclusion

This article has underscored the importance of better understanding the problem of planning for future health workforces and the selection of analysis methods to enable this task. We introduce the notion that a wider range of forecasting methods to support health workforce planning should be considered. As the literature points out, a principal practice of public policy organisations is to argue policy and propose plans in terms of numbers, which acts to reinforce a prediction orientation. However, taking into account the fact that the health workforce is a social system, that we are looking at timeframes longer than the immediate future, and that the problem is wicked in nature, this orientation inevitably produces estimates that are only valid for a short time.

It would be beneficial if these quantitative models were to be complemented by methods that are more attuned to the social dynamics of healthcare and the wickedness of the health planning problem. Our suggestion is that health workforce forecasting needs not only to focus on predicting the numbers of people that will be working in our future health system, but on employing approaches that act as uncertainty reduction mechanisms by describing possible future situations. With these approaches, policy makers are able to consider actions and eventualities as propositions to model future workforce situations, thereby discerning the numbers, roles and skills required. Scenarios are a means to this end and are already being used as part of New Zealand’s health workforce planning system, although there are few observable effects thus far from their development and use. In part to address the core issue cited here, we propose that cultures of certainification in health workforce planning be acknowledged, that we become open to improving problem definitions and that we begin to consider policy identification mechanisms that are more in line with the levels of uncertainty being encountered.

‍